Data Analysis Platform Utilizing Database Relationships to Visualize Data

ABSTRACT

A method generates data visualizations. A computing device retrieves a set of tuples from a database according to user selection. Each tuple has the same set of fields. The device identifies a relation between tuples. The relation is a non-empty set of ordered pairs of tuples from the set of tuples. A user selects a base tuple from the set of tuples and the device forms a filtered subset of tuples consisting of the selected base tuple and those tuples that are connected to the selected base tuple by a sequence of tuples that are related by the relation. The user selects an aggregation level, which consisting of fields from the set of fields. The device generates and displays a data visualization by aggregating the filtered subset of tuples at the selected aggregation level to form a set of aggregated tuples, and displaying each aggregated tuple as a visible mark.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/922,968, filed Jul. 7, 2020, entitled “Data Analysis Platform forVisualizing Data According to Relationships,” which is a continuation ofU.S. patent application Ser. No. 15/724,212, filed Oct. 3, 2017,entitled “Systems and Methods of Arranging Displayed Elements in DataVisualizations that use Relationships,” now U.S. Pat. No. 10,706,061,which is a continuation of U.S. patent application Ser. No. 14/461,331,filed Aug. 15, 2014, entitled “Systems and Methods to Query andVisualize Data and Relationships,” now U.S. Pat. No. 9,779,147, each ofwhich is incorporated by reference herein in its entirety.

This application is related to U.S. patent application Ser. No.14/461,345, filed Aug. 15, 2014, entitled “Graphical User Interface forGenerating and Displaying Data Visualizations that use Relationships,”now U.S. Pat. No. 9,613,086, U.S. patent application Ser. No.14/461,348, filed Aug. 15, 2014, entitled “Systems and Methods forFiltering Data Used in Data Visualizations that use Relationships,” nowU.S. Pat. No. 9,779,150, and U.S. patent application Ser. No.14/461,357, filed Aug. 15, 2017, entitled “Systems and Methods ofArranging Displayed Elements in Data Visualizations that useRelationships,” now U.S. Pat. No. 9,710,527, each of which isincorporated by reference herein in its entirety.

TECHNICAL FIELD

The disclosed implementations relate generally to data visualizationsand more specifically to querying and visualizing both data and datarelationships.

BACKGROUND

Databases are used to track a large amount of data collected during theregular course of business operations and events. Businesses typicallystore data regarding sales and sales projections, profit, inventory,payroll, human resources, and much more. Sports leagues create andmaintain large data warehouses to record scores, standings, andstatistics for every team and every player. As the amount of dataincreases, there is an increasing challenge to extract meaning from thedata. For example, it becomes more difficult to identify hierarchicalstructures, logic patterns, and complicated relationships hidden amongstthe data.

Graphical data visualizations can be effective to convey information andto enable a person to analyze the data. In particular, datavisualizations can aid in human understanding of relationships andpatterns in the data. Many people construct data visualizationsmanually, which is both difficult and time consuming. Data visualizationapplications assist in visualizing data, but many do not supportvisualizing relationships. Some data visualization applications cancreate simple node-link diagrams, are not designed to present complexdata relationships, such as manager reporting structures, productcategories, a social network, family relationships, paper citations, aprogramming class hierarchy, or hyperlinks. Furthermore, datavisualizations with relationships are particularly difficult to presentwhen the amount of data increases.

SUMMARY

Disclosed implementations provide a data visualization engine forvisualizing both data fields and relationships between those datafields. As used herein, the term “relation” may be used interchangeablywith “relationship.” The data visualization engine retrieves a set oftuples from a database according to user selection. Each tuple includesa set of data fields, and in some instances all of the tuples have thesame structure, including number of data fields, order of the datafields, the data types of the data fields, and the data field names. Thedata fields may come directly from fields in the database (e.g., columnsin a database table), or may be computed or derived from one or moredata fields. Each tuple is displayed as a visual mark.

The data visualization engine also displays relationships among theretrieved tuples, using connectors or other visual cues, such aspositioning. In some implementations, a data visualization is furthermodified by other operations, such as filtering, sorting, aggregation ofmarks, or aggregation of connectors.

Although data fields are typically used for the graphical marks, and arelationship is used to create connectors between the marks, someimplementations support using a relationship as a data field or using adata field as a relationship. This flexible architecture enables usersto create data visualizations more quickly and more easily.

A relationship can be encoded in the position of a mark, as a connectordrawn between two marks, or as a property of a mark (e.g. color). Thedirection of the relationship can be encoded by the relative positionsof two marks, by placing an arrowhead on the end of the connector, or bydrawing a connector in a specific way (e.g. using a particular curve).

A relationship can be used to specify the x position or y position ofgraphical marks (e.g., using the row or column shelf, as describedbelow) or for other positional encodings (e.g., radius r or angle θ in apolar layout). A relationship can be combined with a sort order todetermine the location of marks or labels.

A relationship can be used to specify connectors between graphical marks(e.g., edges between nodes), which are drawn as lines or curves betweenthe marks that share the relationship. The type of relationship can beencoded in various properties of the connectors, such as line type orcolor. Properties of a relationship itself can also be encoded asgraphical properties of the connector. For example, the direction of therelationship may be encoded as an arrow head on one side of theconnector or determine how the connector is drawn (e.g. using aparticular curve). A single connector can have multiple encodings (e.g.,size and color). Some implementations support using two or morerelationships simultaneously, and distinct connectors may be displayedusing the multiple relationships. For example, connectors correspondingto different relationships may use different colors.

A connector encoding can work in conjunction with existing datavisualizations that specify the x and y positions of graphical marks. Auser simply adds connectors to the visualization. Connectors can also beused in graphics that do not specify the x and/or y positions of thegraphical marks. In particular, the relationship can be used todetermine positions (e.g., to spread out the nodes in a node-linkdiagram, where the location of the nodes is somewhat arbitrary). It iscommon for a single relationship to be used in multiple ways in a singledata visualization.

When data is aggregated, a pair of tuples may end up having more thanone instance of a relationship because each tuple in the aggregationcould have a different relationship. The number of connections can beencoded in the width or transparency of the connector.

The values of the field(s) used to determine a relationship can be usedfor displaying an associated label. As illustrated in FIG. 4B below, inan equivalence relationship, tuples that share the same value for acertain field are related. That shared value can be used to display anappropriate label. For a first order relationship, the value of a firstfield in one tuple is the same as the value of a second field in asecond tuple. The shared value can be used to display an appropriatelabel for the relationship. In some implementations, labels may beassigned to connectors based on non-shared values (e.g., if connectorsrepresent marriages between people, the connectors could use the firstnames of both people).

Connectors are encoded separately from the marks they are connecting.This means that the connectors keep track of the tuples they areconnecting. A data visualization application looks up the location ofthe graphical marks by their associated tuples in order to connect thedots.

In some implementations, a connector has two or three tuples associatedwith it: the source and destination tuples and an optional relationshiptuple if the relationship is based on two tables. As used herein, theterm “tuple” generally refers to the tuples for the graphical marks, andnot to relationship tuples. Fields in any of these tuples can be used toencode the starting, ending, or overall properties of the connectors.Typically, the source and destination tuples are used to encode startand end properties of each connector, and the relationship tuple encodesthe overall properties of each connector (e.g., color or width of eachentire connector). A relationship tuple is of the form (tuple1, tuple2,[properties]), where tuple1 and tuple2 are tuples for marks that arerelated by the relationship.

An equivalence relationship is slightly different. In general,properties of the connectors may be specified using the properties ofthe two tuples sharing the connection. However, an equivalencerelationship does not have a direction, so there can be ambiguity aboutwhich endpoint tuple to use. Some implementations disallow usingendpoint tuple properties to define graphical properties of connectorswhen the ambiguity is unavoidable. In some implementations, use of suchproperties is allowed, but when there is ambiguity, the encoding doesnot occur. Because an equivalence relationship defines groups ratherthan a direction, some implementations allow connector properties to bebased on the group as a whole (e.g., aggregated properties, such as thenumber of tuples in the group, or the sum or average of some field inthe tuples).

In some implementations, a value of a field for a tuple is used todetermine which point on the mark is used as the connection point.

An alternative to drawing a connector between two marks, especially in adense layout or when the marks are far apart, is to connect to aplaceholder mark that contains information that identifies the othermark it connects to.

As explained in more detail below, a relationship can be used as anordinal field when a sort order has been defined. In some instances, auser defines a relationship, then uses the structure of thatrelationship to define a sort order. This enables a data visualizationapplication to provide more types of sorts. When a relationship has beendefined, a depth-first or breadth-first traversal may create a specificorder, even though it may include some arbitrary traversal decisions. Insome instances, a secondary sort is used to order the children of anode, including the top-level nodes (children of an implicit root).Sorting using a relationship that is not a strict hierarchy may involvedeciding whether or not to allow duplicates in the resulting list.

A connector is drawn between two marks. Marks can get their positionsfrom the row and column selections or from a set of layout algorithmsthat use the row and columns selections as arguments. For example,layouts include radial trees, hyperbolic trees, tree maps, andclustering graph layouts.

When the positions of marks are not the result of specific row andcolumn selection in the user interface, the user may want to move themarks around after they are rendered in a data visualization. Forexample, if a layout algorithm attempts to cluster the marks based onthe various relationships, the user may want to drag some marks to newlocations to help understand the structure.

With connectors, the layout algorithms attempt to limit the amount ofoverlap. However, a user may want to change their routing in variousways to make the connections more obvious, avoid overlap, or emphasize acertain set of relationships. Therefore, a user is generally allowed toalter the location of connectors in a data visualization after it isrendered.

Some implementations provide a group-by shelf, which gives the user theopportunity to provide hints to the layout algorithm for clustering(which affects overall layout). For example, using scores for a sportsleague during a season, a user may suggest grouping by how many timeteams played each other. In the NFL, this would cluster the teams bydivisions, where the teams play each other twice.

The connectors can be drawn in various ways: straight lines, a sequenceof connected orthogonal line segments routed around obstacles, arcs, orother curves. To show the direction of a connector, some implementationsdraw a shape at one or both ends (such as an arrowhead). In someimplementations, direction is indicated by varying properties such assize or color, or by changing the curvature of the arc. Someimplementations allow the user to select how the direction is conveyedin a data visualization.

Relationships are typically binary, tying together two pieces of data.This lends itself well to drawing a connector between two points thatrepresent the two pieces of data. In contrast, an equivalencerelationship is an example of an n-ary relationship (“hypergraph”),tying together an arbitrary number of points. Sometimes this informationis better suited for encoding in the points themselves (e.g., color,shape, or size) than for drawing a connector between every pair ofrelated points. When there are large groups of nodes tied together by anequivalence relationship, the number of connectors grows rapidly (for agroup of n nodes, there are n(n−1)/2 connectors). In this case, oneoption is to draw a single connector from every point in the group to acommon point (which may not be a node). The choice of a common pointcould even add extra information, encoding an average or some othercomputed value.

In a data visualization that includes relationships, there are many waysto filter the data. In one example, a user selects a designated set oftuples, then filters the entire set of tuples to those that have aparticular relationship to one or more of the tuples in the designatedset. For example, limit the set of tuples to those in the designated setplus those tuples that are directly related to one or more of thedesignated tuples. If the tuples represent people, and the relationshipis blood relation, then the filter just described would include aperson's parents and children.

The filtering example just described may be extended by letting the userspecify the number of degrees of separation. In the above example, thenumber of degrees was one. Consider the example of people and theirblood relatives again, and use 2 as the number of degrees of separation(typically this would include 1 degree of separation as well). Twodegrees would include grandparents and grandchildren, but would alsoinclude the person's siblings (children of the person's parents) as wellas other parents of the person's children (generally the person'sspouse).

Filtering of connectors can also be based on aggregation, such as thenumber of connections between two nodes.

Note that filters applied to connectors do not inherently filter thenodes. See, e.g., FIG. 8I below.

Consider a scenario where a relationship has been defined that usesfields in one or more source tables. When the tuple data is aggregated,the specific field values used by the relationship are no longer presentin the result set. Therefore, in order to aggregate relationship data,implementations typically retrieve the entire unaggregated data set.That is, the aggregation is typically performed within the datavisualization application.

For example, consider sports data where one table defines the teams anda second table defines the games the teams have played. See, e.g., FIG.13A. A data visualization may include a mark per team, with connectorsencoding the games they played and connector encodings showing thescores. See, e.g., FIG. 13B. If the teams are aggregated by division,the connector data is typically aggregated as well. See, e.g., FIG. 13C.The connector property aggregation might be total score, average score,etc.

As noted above, data can be aggregated, and marks or connectorsdisplayed based on the aggregated data. A similar process is aggregationof visual marks. Based on the encodings in use, especially discreteencodings, multiple marks could end up mapping to the same location.Likewise, multiple connectors could map to the same location if both endpoints map to the same location. Some implementations support anadditional encoding based on the number of objects that map to the samelocation, which is applied during a consolidation phase after the datahave been retrieved, manipulated, and arranged according to a layoutalgorithm. For example, the size of a consolidated mark may bedetermined by how many marks map to the same location, or the width of aconsolidated connector may be based on how many connectors have endpoints at the same locations. In some implementations, a consolidatedmark or consolidated connector may use the sum of a quantitativeproperty. This feature not only adds useful functionality but speeds uprendering time in some cases.

When there are a limited number of connectors that may connect any pairof nodes, some implementations draw each connector using a differentcurve so that each connector is independently visible.

In accordance with some implementations, a process of generating agraphical representation of a data source is performed at a computerhaving one or more processors and memory. The process generates anddisplays a graphical user interface on a computer display.

In some implementations, “generating” and “displaying” a datavisualization are integrated operations that take raw data from a datasource and a visual specification, and produce visual output on adisplay device. In some implementations, “generating” and “displaying”are separate steps. The generating step takes the raw data and thevisual specification and generates an intermediate output, such as aTIFF, JPEG, PNG, or PDF file, or graphic data formatted in a memorystructure. The display step uses the intermediate output from thegenerating step and displays the data visualization on a display device.In some instances, the term “rendering” is used to identify thegenerating step. When generating and displaying are integrated, one ofskill in the art may use the term “generating” or the term “rendering”to refer to both generating and displaying.

The graphical user interface includes a schema information region and adata visualization region. These may be parts of a single window or inseparate windows. The schema information region includes multiple fieldnames, where each field name is associated with a data field from thedata source. The schema information region also includes one or morerelationship names, where each relationship name is associated with arelationship between rows of the data source. The data visualizationregion includes a plurality of shelves including a row shelf, a columnshelf, and a connector shelf. The process detects a user selection ofone or more of the field names and a user request to associate eachuser-selected field name with a respective shelf in the datavisualization region. The process also detects a user selection of oneor more of the relationship names and a user request to associate eachuser-selected relationship name with a respective shelf in the datavisualization region. The process generates a visual graphic inaccordance with the respective associations between the user-selectedfield names and corresponding shelves and in accordance with therespective associations between the user-selected relationship names andcorresponding shelves, and displays the visual graphic in the datavisualization region.

In some implementations, the visual graphic includes visual markscorresponding to retrieved tuples from the data source. The vertical andhorizontal placement of the visual marks are respectively based on itemsassociated with the row shelf or column shelf respectively by the user.Each item of the items is a field name or a relationship name.

In some implementations, the visual graphic further includes edges thatvisually connect the visual marks, where the edges correspond to arelationship name associated with the connector shelf by the user.

In some implementations, the visual graphic further includes edges thatvisually connect the visual marks, where the edges correspond to a firstfield name associated with the connector shelf by the user. Each edgeconnects two visual marks whose corresponding tuples share a same fieldvalue for the first field name.

In some implementations, a first relationship name is associated withthe column shelf by the user. The horizontal placement of visual marksis determined by a user-selected function of the tuples based on atraversal of a graph corresponding to the tuples and the firstrelationship.

In some implementations, a first field name (of the multiple fieldnames) identifies a computed field whose value for each tuple iscomputed based on an associated data field from the data source and afirst relationship. The first field name is associated with the rowshelf or the column shelf.

In some implementations, the computed value of the computed field foreach tuple is based on a traversal of a graph corresponding to thetuples and the first relationship.

In some implementations, the data visualization region includes one ormore connector property shelves. The connector property shelves mayspecify the color of the connectors or the width of the connectors, asillustrated in FIG. 5A. The connector property shelves may also be usedto specify tapering (e.g., where the width of connectors is wider at oneend point than the other endpoint). In some implementations, one or moreconnector property shelves are used to specify shapes that appear oneach connector (e.g., an arrow at the end of the connector showing thedestination of the relationship).

When the data visualization region includes connector property shelves,in some instances the process detects a user selection of a relationshipname or a field name and a user request to associate the user-selectedrelationship name or field name with a first connector property shelf.In this case, generating the visual graphic includes visually formattingthe connectors in accordance with the user selected relationship name orfield name for the first connector property shelf.

In accordance with some implementations, a process of constructing datavisualizations is performed at a computer having one or more processorsand memory. The process receives a visual specification, which includesa plurality of properties and corresponding user-selected propertyvalues. The properties and property values define the layout of a datavisualization. A first property value of the user-selected propertyvalues identifies one or more source databases for the datavisualization. The process determines one or more node queries from thevisual specification corresponding to one or more data fields in thesource databases. The process also determines one or more link queriesfrom the visual specification corresponding to a first relationshipbetween rows of the source databases. The process retrieves a pluralityof node tuples from the database, where each node tuple satisfies atleast one of the node queries. The process also retrieves a plurality oflink tuples from the database, where each link tuple satisfies at leastone of the link queries. The process generates and displays visual marksin the data visualization corresponding to the retrieved node tuples.The process generates and displays edge marks in the data visualizationcorresponding to the retrieved link tuples. Each edge mark visuallyconnects a pair of visual marks corresponding to the node tuples.

In some implementations, the data visualization is subdivided into aplurality of panes based on the visual specification, where each paneincludes a plurality of visual marks and a plurality of edge marks.

In some implementations, each edge mark connects a pair of visual markswithin a single pane.

In some implementations, at least one edge mark connects a pair ofvisual marks that are in distinct panes.

In some implementations, the first relationship is user-selected from apredefined set of relationships and the one or more link queries areconstructed from the first relationship.

In some implementations, the first relationship corresponds to a datafield f in rows of the source database. Two rows of the source databaseare related by the relationship when the two rows have a same fieldvalue for the data field f.

In some implementations, the first relationship corresponds to a firstfield f and a second field g, both of which are data fields in thesource database. A first row of the source database is related to asecond row of the source database when a field value for field fin thefirst row equals a field value for the field g in the second row.

In some implementations, the one or more link queries are constructedfrom a user selected field in the source database. The link tuplescomprise pairs of rows in the database that have a common value for theuser selected field.

In some implementations, horizontal placement of visual marks isdetermined by a user-selected function of the node tuples based on atraversal of a graph corresponding to the node tuples and a secondrelationship specified by a property in the visual specification.

In accordance with some implementations, a process of filtering data indata visualizations is performed at a computing device having one ormore processors and memory. The process retrieves a set of tuples from adatabase according to user selection, where each tuple includes the sameset of fields. In some implementations, all of the tuples have the samestructure, including number of fields, order of fields, field datatypes, and field names. The process identifies a relationship betweentuples. The relationship is a non-empty set of ordered pairs of tuplesfrom the set of tuples. The process receives selection of one or morefilter conditions for the tuples, where at least one of the filterconditions uses the relationship. The process receives a selection of anaggregation level, which includes one or more fields from the set oftuples. The process generates and displays a data visualization based onaggregating the set of tuples at the selected aggregation level to forma set of aggregated tuples. Each aggregated tuple is displayed as avisible mark. Each tuple that satisfies all of the filter conditions isincluded in an aggregated tuple, and each tuple that fails one or moreof the filter conditions is not included in an aggregated tuple. In someinstances, the process thus uses a relationship between tuples to filterthe displayed set of aggregated tuples without displaying arepresentation of the relationship itself.

In some implementations, the one or more filter conditions include afilter condition that limits the set of tuples to those tuples that areconnected to a selected base tuple. A respective tuple is connected tothe selected base tuple when there is a non-negative integer n and asequence of tuples t₀, t₁, . . . , t_(n) with t₀=the respective tuple,t_(n)=the selected base tuple, and (t_(i-1), t_(i)) is in therelationship for i=1, 2, . . . , n. The special case of n=0 means that abase tuple is considered connected to itself.

In accordance with some implementations, a process of sorting data indata visualizations is performed at a computing device having one ormore processors and memory. The process retrieves a set of tuples from adatabase according to user selection, where each tuple includes a set offields. In some implementations, all of the tuples have the samestructure, including number of fields, order of fields, field datatypes, and field names. The process identifies a relationship betweentuples. The relationship is a non-empty set of ordered pairs of tuplesfrom the set of tuples. The process receives user selection of therelation to specify the x-position or y-position of visual markscorresponding to the tuples. The process generates and displays a datavisualization with each tuple represented by a visible mark. Theposition of each displayed visual mark (x-position or y-position, basedon the user selection) is based on a network traversal of the tuplesusing the relation.

In some implementations, the network traversal uses a depth first searchof the tuples using the relationship.

In some implementations, the network traversal uses a breadth firstsearch of the tuples using the relationship.

In some implementations, the relationship corresponds to a field fin theset of fields. The relationship consists of ordered pairs of distincttuples (t₁, t₂) for which t₁ and t₂ have a same field value for thefield f.

In some implementations, the relationship corresponds to a first field fand a second field g, both in the set of fields. The relationshipconsists of ordered pairs of distinct tuples (t₁, t₂) for which the ffield value for t₁ equals the g field value for t₂.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a context for a data visualization process inaccordance with some implementations.

FIG. 2 is a block diagram of a computing device that a user uses tocreate and display data visualizations in accordance with someimplementations.

FIG. 3 is a block diagram of a data visualization server in accordancewith some implementations.

FIG. 4A illustrates tables in a data source in accordance with someimplementations.

FIG. 4B illustrates various types of relationships in accordance withsome implementations.

FIG. 4C illustrates a table of data used to create family tree diagramsin accordance with some implementations.

FIG. 4D illustrates a pair of tables that illustrates a shipmentrelationship between facilities in different cities in accordance withsome implementations.

FIG. 4E illustrates a category tree hierarchical relation in accordancewith some implementations.

FIG. 5A illustrates a graphical user interface (GUI) that a user may useto create data visualizations in accordance with some implementations.

FIGS. 5B, 5C, and 5D illustrate ways that relationship-based propertiesmay be used in accordance with some implementations.

FIGS. 6A and 6B illustrate high level process flows for creating datavisualizations in accordance with some implementations.

FIG. 7A illustrates Quantitative data fields (Q) and Ordinal data fields(O) in accordance with some implementations.

FIGS. 7B and 7C illustrate some types of data visualizations that may beconstructed according to the type of layout selected by the user and thetypes of data fields selected by the user.

FIG. 7D illustrates some of the data visualizations that may beconstructed according to the user selections for rows and columns.

FIG. 7E illustrates various ways that a relationship can be used as aQuantitative data field in accordance with some implementations.

FIG. 8A provides a chart of data visualizations that include both dataand relationships among the data in accordance with someimplementations.

FIGS. 8B-8M illustrate data visualizations that may be generated anddisplayed based on various user selections for the x-position andy-position of marks in conjunction with user selection of arelationship, in accordance with some implementations.

FIGS. 9A-9C illustrate various data visualizations that include aplurality of marks and connectors in accordance with someimplementations.

FIGS. 10A-10H provide a sequence of data visualizations corresponding toanalysis of airline flights between states in accordance with someimplementations.

FIGS. 11A-11J provide a sequence of data visualizations corresponding tomarket basket analysis of store sales in accordance with someimplementations.

FIGS. 12A-12F provide a sequence of data visualizations corresponding toanalysis of a social network in accordance with some implementations.

FIGS. 13A-13D illustrate some post-rendering interactive datavisualization features that are provided in some implementations.

FIG. 14 identifies some of the ways that relationships can be usedwithin a data visualization in accordance with some implementations.

FIGS. 15A and 15B illustrate using an alternative user interface tocreate group edges in accordance with some implementations.

FIGS. 16A and 16B illustrate using an alternative user interface tocreate path edges in accordance with some implementations.

FIGS. 17A-17E illustrate using an alternative user interface to createedges and nodes based on a relationship in accordance with someimplementations.

FIG. 18 illustrates blending data from two or more non-homogeneous datasources, which may be used as marks or connectors in data visualizationin accordance with some implementations.

FIGS. 19A-19B provide a flowchart of a process, performed at a computer,for generating a graphical representation of a data source in accordancewith some implementations.

FIGS. 20A-20B provide a flowchart of a process, performed at a computer,for generating a graphical representation of a data source in accordancewith some implementations.

FIGS. 21A-21C provide a flowchart of a process, performed at a computer,for filtering data in data visualizations based on a relation inaccordance with some implementations.

FIGS. 22A-22B provide a flowchart of a process, performed by a computer,for sorting data in data visualizations in accordance with someimplementations.

Like reference numerals refer to corresponding parts throughout thedrawings.

Reference will now be made in detail to implementations, examples ofwhich are illustrated in the accompanying drawings. In the followingdetailed description, numerous specific details are set forth in orderto provide a thorough understanding of the present invention. However,it will be apparent to one of ordinary skill in the art that the presentinvention may be practiced without these specific details.

DESCRIPTION OF IMPLEMENTATIONS

FIG. 1 illustrates a context for a data visualization process inaccordance with some implementations. A user 100 interacts with acomputing device 102, such as a desktop computer, a laptop computer, atablet computer, or a mobile computing device. An example computingdevice 102 is described below with respect to FIG. 2, including varioussoftware programs or modules that execute on the device 102. In someimplementations, the computing device 102 includes one or more datasources 236 and a data visualization application 222 that the user 100uses to create data visualizations from the data sources 236. That is,some implementations can provide data visualizations to a user withoutconnecting to external data sources or programs over a network.

However, in some cases, the computing device 102 connects over one ormore communication networks 108 to external databases 106 and/or a datavisualization server 104. The communication networks 108 may includelocal area networks and/or wide area networks, such as the Internet. Adata visualization server 104 is described in more detail with respectto FIG. 3. In particular, some implementations provide a datavisualization web application 322 that runs within a web browser 220 onthe computing device 102. In some implementations, data visualizationfunctionality is provided by both a local application 222 and the server104. For example, the server 104 may be used for resource intensiveoperations while most other operations are performed by the datavisualization application 222 on the device 102.

FIG. 2 is a block diagram of a computing device 102 that a user uses tocreate and display data visualizations in accordance with someimplementations. A computing device 102 typically includes one or moreprocessing units/cores (CPUs) 202 for executing modules, programs,and/or instructions stored in memory 214 and thereby performingprocessing operations; a user interface 206; one or more network orother communications interfaces 204; memory 214; and one or morecommunication buses 212 for interconnecting these components.

The user interface 206 includes a display 208 and one or more inputdevices or mechanisms 210. In some implementations, the inputdevice/mechanism 210 includes a keyboard; in some implementations, theinput device/mechanism includes a “soft” keyboard, which is displayed asneeded on the display device 208, enabling a user to “press keys” thatappear on the display 208. In some implementations, the display 208 andinput device/mechanism 210 comprise a touch screen display (also calleda touch sensitive display).

In some implementations, the communication buses 212 include circuitry(sometimes called a chipset) that interconnects and controlscommunications between system components.

In some implementations, the memory 214 includes high-speedrandom-access memory, such as DRAM, SRAM, DDR RAM, or other randomaccess solid state memory devices. In some implementations, the memory214 includes non-volatile memory, such as one or more magnetic diskstorage devices, optical disk storage devices, flash memory devices, orother non-volatile solid-state storage devices. In some implementations,the memory 214 includes one or more storage devices remotely locatedfrom the CPU(s) 202. The memory 214, or alternatively the non-volatilememory device(s) within the memory 214, comprises a non-transitorycomputer readable storage medium.

The memory 214, or the computer readable storage medium of the memory214, stores the following programs, modules, and data structures, or asubset thereof:

-   -   an operating system 216, which includes procedures for handling        various basic system services and for performing hardware        dependent tasks;    -   a communications module 218, which is used for connecting the        computing device 102 to other computers and devices via the one        or more communication network interfaces 204 (wired or wireless)        and one or more communication networks 108, such as the        Internet, other wide area networks, local area networks,        metropolitan area networks, and so on;    -   a web browser 220 (or other client application), which enables a        user 100 to communicate over a network with remote computers or        devices. In some implementations, the web browser 220 executes a        data visualization web application 322 downloaded from a data        visualization server 104. In some implementations, a data        visualization web application 322 is an alternative to storing a        data visualization application 222 locally;    -   a data visualization application 222, which provides a graphical        user interface (GUI) and enables users to construct data        visualizations from various data sources. In some instances, the        data visualization application 222 retrieves data from a data        source 236 and displays the retrieved data (including        relationships) in one or more data visualizations. In some        implementations, the data visualization application invokes        other modules (either on the computing device 102 or at a data        visualization server 104) to visualize the retrieved data or        relationships. In some implementations, the data visualization        application 222 is a standalone application that runs on the        client device. In some instances, the standalone application 222        retrieves data from a local data source 236, but in other        instances the application 222 retrieves data from a remote        database 106. In some implementations, most of the processing        occurs on the client device, but the data visualization        application 222 hands off certain resource intensive operations        to a data visualization server 104; and    -   one or more data sources 236, which have data fields 238 that        may be displayed by the data visualization application 222. Some        data sources 236 store relationships 240 between other fields.        In some implementations, the relationships 240 are stored        separately from the data fields. Data sources 236 can be        formatted in many different ways, such as spreadsheets, XML        files, flat files, CSV files, text files, desktop database        files, or relational databases. Typically, the data sources 236        are used by other applications as well (e.g., a spreadsheet        application).

In some implementations, the data visualization application 222comprises a plurality of modules. The graphical user interface isprovided by a user interface module 224, which provides the userinterface for all aspects of the application 222. The user interfacemodule 224 is described in more detail below with respect to FIG. 5A.Some implementations include a data retrieval module 226, which buildsand executes queries to retrieve data from one or more data sources 236.The data sources 236 may be stored locally on the device 102 or storedin an external database 106. In some implementations, data from two ormore data sources may be blended. In some implementations, the dataretrieval module 226 uses a visual specification 234 to build thequeries. Visual specifications are described in more detail below withrespect to FIG. 5A.

In some implementations, the data visualization application 222 includesa data visualization generation module 228, which uses retrieved datafrom one or more data sources 236 to generate a data visualizationaccording to the user's request (which may be specified in a visualspecification). The user interface module 224 then displays the rendereddata visualization on the display device 208.

Some implementations include one or more modules to handlerelationships. In some implementations, a relationship identificationmodule 230 automatically discovers some relationships within a datasource 236 (or across data sources 236). For example, the relationshipidentification module may identify an equivalence relationship betweentuples that have the same value for a data field 238 (e.g., for datarepresenting items purchased, two tuples with the same Order ID have therelationship of being in the same order). In some cases, relationshipsare constructed by a user using the relationship builder module 232.Examples of relationships are described in more detail below withrespect to FIG. 4B.

Some implementations use a visual specification 234 to build anddescribe a data visualization. A user builds a visual specification 234implicitly using the user interface, and the visual specification 234specifies what data fields 238 and relationships 240 are used, how theyare encoded, and so on. This is described in more detail with respect toFIG. 5A. The data retrieval module 226 uses the visual specification 234to retrieve the relevant data, and the data visualization generationmodule uses the retrieved data and the visual specification 234 togenerate the data visualization.

In some implementations, the memory 214, or the computer readablestorage medium of memory 214, further stores the following programs,modules, and data structures, or a subset thereof:

-   -   a set of user preferences 242. The user preferences 242 may be        specified explicitly by the user or inferred based on historical        selections by the user; and    -   a data visualization history log 244, which stores data (e.g.,        the data fields and the visual specification) for each data        visualization created by the data visualization application 222.        In some implementations the history log 244 is used to build the        set of user preferences 242.

Each of the above identified executable modules, applications, or setsof procedures may be stored in one or more of the previously mentionedmemory devices, and corresponds to a set of instructions for performinga function described above. The above identified modules or programs(i.e., sets of instructions) need not be implemented as separatesoftware programs, procedures, or modules, and thus various subsets ofthese modules may be combined or otherwise re-arranged in variousimplementations. In some implementations, memory 214 may store a subsetof the modules and data structures identified above. Furthermore, memory214 may store additional modules or data structures not described above.

Although FIG. 2 shows a computing device 102, FIG. 2 is intended more asa functional description of the various features that may be presentrather than as a structural schematic of the implementations describedherein. In practice, and as recognized by those of ordinary skill in theart, items shown separately could be combined and some items could beseparated.

FIG. 3 is a block diagram of a data visualization server 104 inaccordance with some implementations. A data visualization server 104may host one or more databases 106 or may provide various executableapplications or modules. A server 104 typically includes one or moreprocessing units/cores (CPUs) 302, one or more network interfaces 304,memory 314, and one or more communication buses 312 for interconnectingthese components. In some implementations, the server 104 includes auser interface 306, which includes a display device 308 and one or moreinput devices 310, such as a keyboard and a mouse. In someimplementations, the communication buses 312 may include circuitry(sometimes called a chipset) that interconnects and controlscommunications between system components.

In some implementations, memory 314 includes high-speed random accessmemory, such as DRAM, SRAM, DDR RAM, or other random access solid statememory devices, and may include non-volatile memory, such as one or moremagnetic disk storage devices, optical disk storage devices, flashmemory devices, or other non-volatile solid state storage devices.Memory 314 may optionally include one or more storage devices remotelylocated from the CPU(s) 302. Memory 314, or alternately the non-volatilememory device(s) within memory 314, comprises a non-transitory computerreadable storage medium.

In some implementations, memory 314 or the computer readable storagemedium of memory 314 further stores the following programs, modules, anddata structures, or a subset thereof:

-   -   an operating system 316, which includes procedures for handling        various basic system services and for performing hardware        dependent tasks;    -   a network communication module 318, which is used for connecting        the server 104 to other computers via the one or more        communication network interfaces 304 (wired or wireless) and one        or more communication networks 108, such as the Internet, other        wide area networks, local area networks, metropolitan area        networks, and so on;    -   a web server 320 (such as an HTTP server), which receives web        requests from users and responds by providing responsive web        pages or other resources;    -   a data visualization web application 322, which may be        downloaded and executed by a web browser 220 on a user's        computing device 102. In general, a data visualization web        application 322 has the same functionality as a desktop data        visualization application 222, but provides the flexibility of        access from any device at any location with network        connectivity, and does not require installation and maintenance.        In some implementations, the data visualization web application        322 includes various software modules to perform certain tasks,        including a user interface module 224, a data retrieval module        226, a data visualization generation module 228, a relationship        identification module 230, and a relationship builder module        232. These software modules are described above with respect to        FIG. 2, and are described in more detail below. In some        implementations, the data visualization web application 322 uses        a visual specification 234, as described above with respect to        FIG. 2 and described below with respect to FIG. 5A; and    -   one or more databases 106, which store data used or created by        the data visualization web application 322 or data visualization        application 222. The database 106 may store data sources 236,        which provide the data used in the generated data        visualizations. A data source 236 may store data in many        different formats, and commonly includes many distinct tables,        each with a plurality of data fields 238. Some data sources        comprise a single table. The data fields 238 include both raw        fields from the data source (e.g., a column from a database        table or a column from a spreadsheet) as well as derived data        fields, which may be computed or constructed from one or more        other fields. For example, derived data fields include computing        a month or quarter from a date field, computing a span of time        between two date fields, computing cumulative totals for a        quantitative field, computing percent growth, and so on. In some        instances, derived data fields are accessed by stored procedures        or views in the database. In some implementations, the        definitions of derived data fields 238 are stored separately        from the data source 236. In some implementations, the database        106 stores relationships 240 identified by relationship        identification module 230 or constructed by the relationship        builder module 232. For example, relationships built by one user        100 may be subsequently used by other users. In some        implementations, the database 106 stores a set of user        preferences 242 for each user. The user preferences may be used        when the data visualization web application 322 (or application        222) makes recommendations about how to view a set of data        fields 238. In some implementations, the database 106 stores a        data visualization history log 244, which stores information        about each data visualization selected by the user 100. In some        implementations, the database stores other information,        including other information used by the data visualization        application 222 or data visualization web application 322. As        illustrated in FIGS. 1 and 3, databases 106 may be separate from        the data visualization server 104, or may be included with the        data visualization server (or both).

In some implementations, the data visualization history log 244 storesthe visual specifications selected by users, which may include a useridentifier, a timestamp of when the data visualization was created, alist of the data fields used in the data visualization, the type of thedata visualization (sometimes referred to as a “view type” or a “charttype”), data encodings (e.g., color and size of marks), the datarelationships selected, and what connectors are used. In someimplementations, one or more thumbnail images of each data visualizationare also stored. Some implementations store additional information aboutcreated data visualizations, such as the name and location of the datasource, the number of rows from the data source that were included inthe data visualization, version of the data visualization software, andso on.

Each of the above identified executable modules, applications, or setsof procedures may be stored in one or more of the previously mentionedmemory devices, and corresponds to a set of instructions for performinga function described above. The above identified modules or programs(i.e., sets of instructions) need not be implemented as separatesoftware programs, procedures, or modules, and thus various subsets ofthese modules may be combined or otherwise re-arranged in variousimplementations. In some implementations, memory 314 may store a subsetof the modules and data structures identified above. Furthermore, memory314 may store additional modules or data structures not described above.

Although FIG. 3 shows a data visualization server 104, FIG. 3 isintended more as a functional description of the various features thatmay be present rather than as a structural schematic of theimplementations described herein. In practice, and as recognized bythose of ordinary skill in the art, items shown separately could becombined and some items could be separated. In addition, some of theprograms, functions, procedures, or data shown above with respect to aserver 104 may be stored on a computing device 102. In someimplementations, the functionality and/or data may be allocated betweena computing device 102 and one or more servers 104. Furthermore, one ofskill in the art recognizes that FIG. 3 need not represent a singlephysical device. In some implementations, the server functionality isallocated across multiple physical devices that comprise a serversystem. As used herein, references to a “server” or “data visualizationserver” include various groups, collections, or arrays of servers thatprovide the described functionality, and the physical servers need notbe physically collocated (e.g., the individual physical devices could bespread throughout the United States or throughout the world).

FIG. 4A illustrates a data source 236 with four tables, which may bestored in a database 106, such as a structured query language (SQL)database. The data source 236 organizes the data into tables where eachrow corresponds to a basic entity or fact and each column represents aproperty of that entity. For example, a table may represent transactionsat a bank, where each row corresponds to a single transaction and eachtransaction has multiple attributes (data fields 238), such as thetransaction amount, the account balance, the bank branch, and thecustomer. FIG. 4A illustrates an exemplary data source 236 that includesa base table 402 and a plurality of lookup tables 404, 406, and 408 inaccordance with some implementations.

In this example, the base table 402 represents sales data for a businessentity, where each row corresponds to certain sales information for aspecific product. Each row of the base sales table 402 has multipleproperties, including the store, the month, the product, the scenario,the sales, and the costs. As used herein, a row in a table is commonlyreferred to as a tuple or record, and a column in a table is referred toas a data field 238. The base table 402 and the plurality of lookuptables 404-408 together form a star schema in which the central facttable is surrounded by each of the dimension tables that describe eachdimension (or attribute) of the central fact table. In this example, thebase sales data table 402 is the fact table and each lookup table is adimension table.

The data fields 238 within a table can be categorized in various ways.In some implementations, each data field 238 is classified as either a“dimension” or a “measure.” Dimensions and measures are similar toindependent and dependent variables in traditional analysis. In abanking example, the bank branch and account number are dimensions (theyare independent), whereas the account balance is a measure (it dependson the branch and account selected). A single database will oftendescribe many heterogeneous but interrelated entities. For example, adatabase designed for a coffee chain might maintain information aboutemployees, products, and sales.

Some implementations also classify data fields 238 based on their datatypes. Although there are many different data types used by various datasources 236 (e.g., 16-bit integer, 32 bit integer, single precisionfloating point, double precision floating point, fixed size decimal,date/time, fixed length character string, variable length characterstring, Boolean, etc.), it is useful to classify these data types basedon the structure of their values. In some implementations, each datafield 238 is classified as ordinal (O) or quantitative (Q). The valuesof an ordinal data field 238 are discrete, typically corresponding todata values that are character strings (e.g., regions). The values of aquantitative data field 238 are continuous, such as sales or profit. Theclassification and use of ordinal and quantitative data fields isdescribed in more detail below with respect to FIGS. 7A-7D.

Disclosed implementations visualize not only tuples of data fields 238,but also relationships between tuples. For example, visualizing a socialnetwork may include a node for each person (each person corresponding toa tuple) and connectors between nodes to depict relationships betweenpeople in the social network. FIG. 4B identifies some of the types ofrelationships that may be established and visualized. In someimplementations, these relationships are identified by the relationshipidentification module 230 or constructed by a user 100 using therelationship builder module 232.

In some implementations, a first-order relationship 410 is identifiedwhen a value of a first data field 238 of a first tuple is equal to avalue of a second data field 238 of a second tuple. One example of thisis illustrated in FIG. 4C.

FIG. 4C illustrates a simplified family tree table 438, which includesdata for various people. In this example, each person is uniquelyidentified by an ID 440. The table 438 also includes each person's firstname 442 and last name 444. Some family tree tables 438 address the factthat some names change over time, but in this example the datarepresents a person's first and last name at birth. Even if two peopleshare the same first and last names, they would have distinct ID values.The family tree table may include various data about the people,including gender 452, birth_date 454, and birth_place 456. In thissimplified example, the birth_place 456 may be limited to U.S. states orforeign countries. In some instances, the family tree table 438 includesdata fields to track when a person dies (death_date 458) and thelocation where the person died (death_place 460). In someimplementations, the death_date 458 and death_place 460 are NULL orblank while the person is still alive.

In addition to this basic data about each person, this sample familytree table 438 includes information that shows relationships with otherpeople. When specified, the father_id 446 is the ID number of a person'sfather. For example, Bob's father_id is 1, which is the ID of Abe, sothis information shows that Abe is Bob's father. Similarly, themother_id field 448, when present, specifies the ID of a person'smother. For example, both Dave and Edith have mother_id=3, withspecifies that their mother is Cathy Smith. Finally, for those peoplewho are or were married, the table 438 includes a spouse_id 450, whichspecifies the ID of a person's spouse. In this example, Cathy (ID=3) isthe spouse of Bob (ID=2), and vice versa. Note that the father_id andmother_id are permanent facts, whereas a person could remarry afterdivorce or the death of an earlier spouse. Some implementations of afamily tree table support these more complex scenarios.

The “child-father” relationship created by this family tree table 438 isa first-order relationship 410. In this example, both the person and theperson's father are tuples in the same table 438.

FIG. 4D illustrates another example of a first order relationship. FIG.4D includes a facilities table 470 that defines facilities. This highlysimplified table 470 includes a unique facility_id 472 for each facilityand two pieces of information about the facility: the city 474 where thefacility is located and its capacity 476, which is some relevant measureof volume (e.g., cubic feet, cubic yards, bushel boxes, or TEU(twenty-foot equivalent units, used in the container industry)). FIG. 4Dalso includes a shipment table 480, which identifies shipments betweenthe facilities. Each shipment record is uniquely identified by ashipment_id 482. The shipment table 480 includes various informationabout each shipment, including the ship_date 484, the receive_date 486,the item shipped 494, the amount 492 of the item shipped, the cost 496of the shipping, and the carrier 498. The amount 492 is typicallyspecified using the same unit of measure as the capacity data field 476in the facilities table 470. One of skill in the art will recognize thatan actual shipment table 470 would contain much more information, suchas the weight, the volume, the monetary value, the number of widgets,the number of boxes, the mileage, and so on.

The shipment table has a “from” field 488, which specifies a facility_id472 for the starting point of the shipment and a “to” field 490, whichspecifies the ending point of the shipment. The shipment table 480 inthis example creates a relationship between the facility tuples. Inparticular, the origin is the facility tuple where the value of thefacility_id 472 matches the value of the “from” field 488 in theshipment table. The destination is the tuple where facility_id matches“to” field 490. The shipment table is the relationship table, whichallows for properties on the relationship itself. Some implementationsuse the notation {{facility_id=from}={facility_id=to}} to represent thisrelationship. This is another example of a first order relationship 410.

Note that in these two first order relationships, the roles of thetuples is not symmetric. In the first example, Abe is the father of Bob,but Bob is not the father of Abe. Similarly, in the second example, ashipment going from Seattle to San Diego is quite different from ashipment in the opposite direction. In some implementations, once arelationship is defined, an inverse relationship may also be used. Aninverse relationship uses the same tuples, but has the opposite“direction” (e.g., “received from” would be the inverse of “shipped to”in the second example above).

In some implementations, a second order relationship 412 is created bychaining together two first-order relationships (which may be the samerelationship). For example, a “paternal grandfather” relationship couldbe defined as one in which the father field of one tuple matches theperson id field of a second tuple and the father field of the secondtuple matches the person id field of a third tuple. The third tuplespecifies the paternal grandfather of the first tuple. In someimplementations, this relationship uses the notation {{ID=father_id},{ID=father_id}}. Higher order relationships 412 can be defined in asimilar fashion.

Some implementations also allow n-order relationships to be combinedinto a more complex relationship 414. For example, consider a parentrelationship, expressed as {id=father|mother}. That is, the person ID ofthe second tuple matches either the father or mother fields of the firsttuple. The grandparent relationship can be expressed as{{id=father|mother}, {id=father|mother}}, and so on. A descendantrelationship can be defined as the union of the first order parentrelationship {id=father|mother}, the second order grandparentrelationship {{id=father|mother}, {id=father|mother}}, the third ordergreat-grandparent relationship, and so on. This chaining of one or morefirst order relationships in this way can be represented as{id=father|mother}*, where the asterisk * indicates one or moreiterations of the first order relationship. This is an example of arelationship 414 defined as a union of chained first orderrelationships.

An equivalence relationship 416 is a relationship between tuples thatshare the same value for a specified data field. (Some more complexexamples are described below.) For example, in a database of people,there is an equivalence relationship between people who share the samelast name. Some implementations express this as {last name}. In someinstances, the equivalence relationship requires two or more fields fromthe tuples to have matching field values. For example, suppose a largeretailer collects sales data from many stores. Each store has a uniquestore ID, and each order at a store has a unique order ID. Each ordermay have multiple line items. Each store operates independently of theothers, so the same order IDs may be used at different stores. On aweekly basis, all of the sales data is collected from all of the storesinto a single data warehouse. Within this data warehouse, an equivalencerelationship is created to group items that were purchased together in asingle order. In this case, tuples must have the same store ID and thesame order ID in order to be related. This equivalence relationship isexpressed as {{store ID} & {order ID}} in some implementations. Moregenerally, {{field1} & {field2}} may be used to denote an equivalencerelationship that requires two matching fields. The same notation can beextended to three or more fields.

In some instances, tuples are related when either of two fields havematching values. For example, the tuples may include the data fieldsfield1 and field2. If two tuples have matching data values for field1,then the tuples satisfy the equivalence relationship. On the other hand,if two tuples have matching data values for field2, the two tuplessatisfy the equivalence relationship as well. Matching either one of thedata fields field1 or field2 (or both) establishes the relationship.Some implementations use the notation {{field1}|{field2}} for thisrelationship. For example, in a table of people, a “sibling”relationship could be defined as those individuals who share a commonmother or father (or both). This could be expressed as{{mother}|{father}}. The same relationship concept can be extended tothree or more fields. In addition, “and” and “or” operations can becombined in many other ways to create more complex equivalencerelations.

A delta-tolerance relationship 418 is defined using a quantitative datafield 238 and a positive tolerance value Δ. For example, suppose eachtuple has the quantitative data field X, suppose a and b are two suchtuples, and suppose a tolerance value Δ=0.35 is specified. Then the pairof tuples (a, b) satisfies the relationship if |a·X−b·X|<0.35. Note thatthis delta-tolerance relationship 418 is not an equivalence relationship416 because a delta-tolerance relationship 418 is not transitive. One ofskill in the art recognizes that delta-tolerance relationships can beexpanded in various ways by including two or more data fields 238 in thecalculation or forming a Boolean combination of two or moredelta-tolerance calculations.

Some implementations support clustering relationships 420. One of skillin the art recognizes that various clustering algorithms can be appliedto one or more of the quantitative data fields in the retrieved tuples,which results in partitioning the tuples into a plurality of distinctclusters. For example, suppose there are two distinct quantitative datafields 238 in the tuples, and these two quantitative fields will be usedto specify the x-position and y-position of marks in a scatter plot. Insome instances, the data naturally subdivides into distinct clusters asseen in the scatter plot. In this case, a clustering relationship 420can be defined based on the clusters. That is, every pair of tupleswithin a cluster is related, and no tuple is related to a tuple in adifferent cluster.

One of skill in the art will recognize that many other types ofrelationships can be identified or constructed, and these relationshipsmay be either identified by the relationship identification module 230or constructed using the relationship builder module 232. Once arelationship is constructed, some implementations classify therelationship. FIG. 4B illustrates some of the classifications.

Some relationships are classified as directed relationships 422. In theexamples above, the first order relationships 410, the second orderrelationships 412, and higher order relationships 414 are all directed.For example, in a child-father relationship, the roles of the child andthe father are not interchangeable. In the example above of shipmentsfrom one facility to another, the relationship is directed because theshipments take products from a source facility to a destinationfacility.

On the other hand, some relationships are classified as undirectedrelationships 424. For example, an equivalence relationship 416 isundirected. If tuple A is related to tuple B, then tuple B is related totuple A. Clustering relationships 420 are similarly undirected. Whenevertwo tuples are in the same cluster, they are related. That is, if tupleA is in the same cluster as tuple B, then tuple B is in the same clusteras tuple A.

In some instances, an undirected relationship 424 can be converted intoa directed relationship 422 by assigning a “direction” (potentiallyarbitrarily) to each relation between pairs of tuples. In otherinstances, a directed relationship 422 may be converted to an undirectedrelationship 424 by ignoring the direction of the original relations.

Some implementations identify whether a relationship has any loops(e.g., tuple A is related to tuple B, tuple B is related to tuple C, andtuple C related to tuple A). A relationship without any loops may beclassified as a tree 426, whereas a relationship with one or more loopsis typically classified as a graph 428. Although the term “tree”commonly refers to a graph that is fully connected (and acyclic), asused herein, a tree may consist of multiple disconnected portions, aslong as there are no cycles.

FIG. 4E illustrates a relationship between tuples based on a data fieldhierarchy. This is sometimes referred to as a category tree. One aspectof these relationships that is different from those depicted in FIG. 4Bis that additional tuples are automatically constructed as needed tofill out the hierarchy. The example in FIG. 4E is based on collegefootball teams in the United States. Each team 570 is assigned to aspecific division 572 and a specific subdivision 574 within the division572. As illustrated in the team hierarchy 564, there is a hierarchyrelationship between divisions, subdivisions, and teams. However, eachof the tuples in the team table 562 (e.g., tuples corresponding to rows576, 578, 580, and 582) represents a team 570. There are no tuples thatrepresent a division or subdivision. For example, the data in the firstrow 576 is for the team Boston College. The row 576 identifies thedivision 572 and subdivision 574 that Boston College belongs to, but therow 576 does not represent an entire division or subdivision. Inparticular, there are other rows, such as the second row 578 that belongto the same division and subdivision.

In this case, when the defined relationship is used, additional tuples566 are added to represent the divisions and subdivisions. Note that arow for a division, such as row 568 for the Atlantic Coast division,only has the name of the division, and no other data, because the otherdata fields team 570, subdivision 574, etc. are not properties of adivision. On the other hand, the additional rows for subdivisionsinclude the information that specifies which division they are in. Asused in this disclosure, the term “row” typically refers to rows fromthe data source 236, whereas “tuple” typically refers to a record thathas been retrieved from the data source, and potentially modified invarious ways. For example, a retrieved tuple only includes the datafields that are needed for the requested data visualization, which istypically fewer than all of the fields in the data source. In addition,the tuples may include additional computed data fields. Here, there areadditional tuples to fill out the hierarchy.

Having defined the category tree relationship, a user could constructthe team hierarchy 564 using the user interface 500. For example, theconnectors are specified by the relationship, and the positioning of theelements uses one or more quantitative fields constructed from therelationship, similar to those described below with respect to FIG. 7E.In particular, a category tree forms a tree relationship R_(T) 426 amongthe tuples once the additional tuples 566 are added, so a depth firstsearch of the tree creates an ordering for the tuples as illustrated inthe team hierarchy 564. The divisions 584 and 596 are the top level“roots” for each subtree. In this illustration, the Big East division596 is not divided into subdivisions, so all of the teams, includingWest Virginia 598 (corresponding to row 582 in the data source) areordered under the division 596. The Atlantic Coast division 584 isdivided into the Atlantic subdivision 586 and the Coastal subdivision592, and each of these subdivisions has multiple teams. In this example,Boston College 588 and Clemson 590 are in the Atlantic subdivision 586and Virginia Tech 594 is in the Coastal subdivision 592. In this case,the teams within a subdivision may be ordered by a specific rule (e.g.,alphabetical, by rank, etc.), or it may be arbitrary.

FIG. 5A illustrates a graphical user interface (GUI) 500 that enablesusers to build data visualizations in accordance with someimplementations. The user interface 500 includes a schema informationregion 510 and a data visualization region 520. The data visualizationregion 520 includes a plurality of shelves (described below) where theuser 100 specifies various characteristics of a desired datavisualization. Below the shelves in this implementation, the datavisualization region 520 includes a graphic display region 530 where thedata visualizations are displayed. In this illustration, the graphicdisplay region 530 is displaying a data visualization 540.

The schema information region 510 displays the data fields 238 andrelationships 240 from the selected data source(s) 236. In someimplementations, the schema information region 510 is subdivided intosections. In this illustration, the schema information region 510includes a dimensions section 502, a measures section 504, and arelationships section 506. As noted above, some implementationscategorize the data fields 238 into dimensions and measures. Someimplementations then display the dimension data fields 238 in thedimensions section 502 and the measure data fields 238 in the measuressection 504. Each data field 238 may correspond directly to a field inthe underlying data source 236, or may be computed or derived from oneor more data source fields. In the example depicted in FIG. 5A, themeasures section 504 includes three derived data fields 238 that arebased on the child-father relationship 240 (i.e., the three data fieldsat the bottom of the measures section 504, beginning with the deriveddata field Overall({ID=father_id}) 238. In the dimensions section 502,the Life Span data field 238 is computed based on the birth year anddeath year, which are derived from the Birth Date and Death Datemeasures. Another example of a derived data field 238 is age. In someinstances, “age” is the difference between the current date (or somereference date) and a birth date or other start date. Using the datafields 238 in the schema information region, age at death may becomputed by subtracting the birth date from the death date (andtypically truncating to full years).

In some implementations, the dimensions and measures are groupedtogether in a single section. In some implementations, the dimensionsand measures are sorted alphabetically. In some implementations, a usercan assign an alias or label to a data field 238, which can be usedinstead of the field name from the data source 236. In someimplementations, a user may specify a sort order for the dimensions ormeasures. This may be particularly useful when the number of data fields238 requires scroll bars in the dimensions section 502 or measuressection 504 (e.g., sort the data fields so that the most frequently usedfields are at the top).

In addition to the data fields 238, the schema information region 510includes a relationships section 506, which displays zero or morerelationships 240 among the data (e.g., relationships such as thoseillustrated in FIG. 4B). Some of the relationships 240 may be identifiedby the relationship identification module 230 (e.g., relationships basedon a single field or relationships previously defined by the same or adifferent user). In some implementations, the user interface 500includes a button, toolbar icon, or menu option to invoke therelationship builder module 232, which can be used to define one or morerelationships. Once constructed, such a relationship displays in therelationship section 506, and may be saved (e.g., in database 106) forfuture use. In some implementations, relationships 240 are assigned adefault name or label, such as “{ID=father_id}” as illustrated in therelationships section 506. In some implementations, a user 100 mayassign an alternative name or label to relationships.

As illustrated in FIG. 5A, data fields 238 are used to buildrelationships 240, and relationships can be used to construct additionaldata fields. Here, the data fields ID and father_id are used to buildthe father relationship {ID=father_id}. In addition, the relationship{ID=father_id} is used to build three data fields 238. The data fieldlabeled Overall({ID=father_id}) represents an overall position of eachtuple in a traversal of a graph formed using the {ID=father_id}relationship. (See also FIG. 7E.) The data field labeledDepth({ID=father_id}) represents the depth of each tuple in a treeformed by the relationship. Similarly, the data field labeledBreadth({ID=father_id}) can be defined for each tuple as the totalnumber of tuples at the same depth. In some instances, the number of“siblings” includes the tuple itself. Mathematically, the breadth attuple A can be computed by finding the unique tuple B (if one exists inthe data set), for which A.father_id=B.ID, then computing thecardinality of the set of all tuples C for which C.father_id=B.ID. Insome instances, the breadth at a tuple defaults to 1 when the tuple hasno father in the data set. In some instances, the breadth of each tuplecan be defined as the number of children of each tuple. In other words,for a tuple A, the number of children is the cardinality of the set{B|B.father_id=A}.

Each of the data fields 238 and each of the relationships 240 can beassociated with a shelf in the data visualization region 520. Somesystems to create data visualizations are described in U.S. Pat. No.7,089,266, entitled “Computer Systems and Methods for the Query andVisualization of Multidimensional Databases,” which is incorporatedherein by reference in its entirety. Additional data visualizationsystems are described in U.S. Pat. No. 8,099,674, entitled “ComputerSystems and Methods for Automatically Viewing MultidimensionalDatabases” and U.S. patent application Ser. No. 14/242,843, filed Apr.1, 2014, entitled “Systems and Methods for Ranking Data Visualizations,”both of which are incorporated herein by reference in their entirety.

In some implementations, the data visualizations are broadly categorizedinto general layout types. Some implementations include the layout typesCartesian, circular (also referred to as “radial” or “polar”), packed,and network. In Cartesian layouts, the rows shelf 532 and the columnsshelf 534 typically define the y-position and x-position of graphicalmarks (e.g., points in a scatter plot or bars in a bar chart). The typeof marks in a data visualization is generally associated with the viewtype (also referred to as a “chart type”). For example, bars in a barchart, text in a text table, points in a scatter plot or line chart, andso on.

The visual appearance of the marks can be modified by various encodings.In the text encoding shelf 542, the user may specify one or more datafields whose text will be used with the marks (either as the marksthemselves, or as associated labels). In the example of FIG. 5A, thetext encoding specifies the “Name” and “Life Span” data fields 238, sothe text for these two fields are used as part of the marks in the datavisualization 540.

The color encoding shelf 552 can be used to specify different colors formarks based on values of a data field. For example, a user could selectthe “Gender” data field for color encoding, in which case the datavisualization 540 would use different colors for men and women. In thisexample, color encoding may be implemented as the background color foreach of the boxes, so the background color for Edith would be differentfrom the other four boxes. Implementations typically assign certaincolors by default, but the default colors may be changed by the user.

The size encoding shelf 544 may be used to correlate the size of visualmarks with a quantitative data field. In the example of FIG. 5A, sizeencoding could be used to identify some relevant characteristic of thepeople. For example, the size could be correlated with the number ofchildren for each person. In other contexts, size could be used toillustrate each person's age, height, wealth, or other characteristicsthat are numeric in nature.

The shape encoding shelf 554 can be used to specify different shapes formarks based on a data field. For example, if the Gender data field wereused for shape encoding, men and women could be differentiated by theshapes of the marks.

Note that the shelves 542, 544, 552, and 554 identify encodings for thegraphical marks. Some implementations enable more or fewer encodingshelves. Some implementations impose limits on which data fields 238 maybe selected for each of the encoding types. For example, size encodingis typically limited to quantitative data fields (e.g., size encodingbased on Birth Place would not make sense). Because shape encodingbecomes ineffective when there are too many shapes, some implementationslimit shape encoding to data fields with less than a predefined maximumnumber of distinct values (e.g., setting the maximum number as 10 or15).

In some implementations, each mark includes a text indicator, a graphicmark, or both. The visual encodings of the marks may include displayedtext, text color, text size, mark shape, mark size, mark color, or otherencodings.

With layout types other than Cartesian, the encoding shelves typicallyoperate in the same way. However, the rows and columns shelves 532 and534 have different meanings. Some examples with circular layouts areillustrated in FIG. 7B below. In some implementations, the labels “Rows”and “Columns” are changed dynamically based on the layout type and/orthe view type in order to clarify how the shelves are used. In someimplementations, when circular layouts are used, the rows shelf 532 andthe columns shelf 534 define the radial axis and the angular axis of thecorresponding polar diagrams, somewhat like using polar coordinates.

In addition to the graphical marks, implementations support connectorsthat connect graphical marks together. The connectors are sometimesreferred to as “edges” or “edge marks.” The connector shelf 536 is usedto specify which graphical marks (corresponding to tuples) aregraphically connected. Typically, a relationship 240 is used to specifythe connectors, such as the relationship {ID=father_id} illustrated onthe connector shelf 536 in FIG. 5A. This specifies that a connector willbe drawn between pairs of marks when the child-father relationship 240holds between the tuples. The connector lines 560 connect thecorresponding visual marks. Although the connector lines 560 here aredrawn as straight lines, some implementations use arcs or other curvesas connectors. In some implementations, the shape of the connector lines560 is configurable, either globally or for individual datavisualizations.

A user 100 may also place a data field 238 on the connector shelf 536.When a data field 238 is placed on the connector shelf 536, it istreated as an equivalence relation 416, in which tuples with the samevalue for that data field are connected. In particular, the user canplace a data field 238 on the connector shelf, and is not required tobuild an equivalence relationship (using the relationship builder module232) first. For example, in a data source 236 representing purchasetransactions, placing the “order ID” data field 238 on the connectorshelf would connect together each pair of items from the same order.

Some implementations allow more than one data field 238 or relationship240 to be placed on the connector shelf 536. When the connector shelf236 contains two or more items, implementations must address the issueof whether to use AND or OR to combine the items. In someimplementations, there is a default behavior, which is displayed, andthe user can change that default behavior. In some implementations, thedefault behavior is to treat multiple items on the connector shelf 236with OR, so that two marks are connected if any of the relationshipsapply. For example, if the user 100 dropped the additional relationship{ID=mother_id} on the connector shelf 536, the connector shelf maydisplay {ID=father_id} OR {ID=mother_id}. In this case, there is aconnector between each child and parent. When there are three or moreitems on the connector shelf 536, parentheses may be required to specifythe desired combination of relationships (e.g., “(A ORB) AND C”). Insome implementations, the relationship builder module 232 provides aninterface that can construct new relationships from data fields 238 aswell as construct new relationships 240 from existing relationships 240.In some implementations, when an expression with two or more items isused on the connector shelf, the user has the option of saving thecombined relationship as a single new relationship. For example, thecombined relationship {ID=father_id} OR {ID=mother_id} could be savedand named “Parent” for future use.

Like the graphical marks, connectors 560 have various encodings. In someimplementations, there are encodings for line style, line size, linecolor, whether the lines have arrows or other shapes where they connectto the marks, line tapering (non-constant line width), ordering, andothers. The user interface 500 illustrated in FIG. 5A includes encodingsfor size and color.

The encodings for connectors introduces some complexity not present inthe encodings for the graphical marks. For example, if a certain datafield 238 is used to encode size or color of connectors 560, whichendpoint of the connector is used to define the data field value used?As a specific example, suppose a user selects the Gender data field 238to encode the color of connectors. The connector 560 between Edith andBob could be either color because one endpoint of the connector has thevalue “female” for the Gender field, whereas the other endpoint has thevalue “male” for the Gender field. Implementations address thisambiguity in various ways. In some instances, such as an equivalencerelationship 416, the two endpoints are guaranteed to have one or moredata fields with the same value. When the data field values areguaranteed to be the same, there is no encoding problem. Forrelationships that are directed, some implementations allow specifyingwhich endpoint is used. For example, the relationship {ID=father_id} isdirected, so a user may specify the head or tail of the relationship forsize or color encoding (e.g., when placing a data field onto theconnector size encoding shelf 546 or the connector color encoding shelf556, the user 100 may be prompted to select head or tail). In someimplementations, this is depicted on the size shelf 546 or color shelf556 as “Gender (from)” for example. Some implementations include aconnector shape encoding shelf that can be used to specify properties ofa connector that show direction (e.g., shape of arrowhead).

Note that a tapering encoding for connectors 560 does not have the sameambiguity problem. If a quantitative data field 238 is selected fortapering, then the values of the quantitative field at both endpointsare used, and the connector between them tapers based on the differencebetween the two values. Some implementations restrict what encodings maybe used to avoid ambiguity.

In some instances, a relationship 240 itself has associated data. Forexample, the first order relationship 410 illustrated with respect toFIG. 4D has many different properties that are associated directly witheach shipment (e.g., amount 492, item 494, and cost 496). In someimplementations, the Relationships section 506 includes therelationship-based properties 508, as illustrated in FIG. 5B. When thereare relationship-based properties 508, they may be used for theconnector encodings within typical constraints (e.g., the “carrier”property would be appropriate for color encoding, but not appropriatefor size encoding). Note that the relationship-based properties aregenerally not available for the encoding of marks. In general, suchencoding would not make sense, and it would also be ambiguous because asingle mark may be related to two or more other marks. For example,using the data in FIG. 4D, it would not make sense to encode marks forfacilities based on properties of shipments to or from the facilities.

Instead of identifying the relationship-based properties 508 in theRelationships section 506 of the schema information region 510, someimplementations allow a user to place a relationship on a connectorencoding shelf (such as the size shelf 546 or the color shelf 556), andprompt the user to select a specific relationship-based property 508 atthat time. Some implementations use a pop-up window such as the oneillustrated in FIG. 5C.

As described below (e.g., FIGS. 10A-10H and 11A-11J), connectors 560 maybe encoded based on aggregated data as well. Although not illustrated inFIG. 5A, a user may specify a level of detail for the graphical marks,in which case a single graphical mark may represent an aggregation ofmultiple records from the data source 236. Although aggregation may notbe as useful for the family tree data illustrated in FIG. 5A,aggregation is frequently useful for business data. When the graphicalmarks are based on an aggregation of tuples, the connectors betweenthose tuples may be aggregated as well. For example, in FIGS. 11A and11B, the size of the graphical marks is based on the number of tuplesaggregated. Similarly, the size (or color) of connectors can be computedbased on aggregating data from the tuple-level relationship. Forexample, in FIG. 10E, the size of the connectors is based on the numberof flights between each of the states.

In addition to the count of the number of individual relationshipsbetween tuples, some data sources 236 are suitable for aggregation bysumming or averaging. For example, in the shipping example of FIG. 4D, arelationship between facilities is defined based on shipments betweenthose facilities. Associated with the shipments are the amount 492 andthe cost 496. If the connector data is aggregated, the amount 492 or thecost 496 could be summed or averaged and used for a size or colorencoding

Consider the following example that uses the data from FIG. 4D. Assumethat visual marks are displayed for each facility, and assume there isshipment data for a year (or is filtered to a specific year). Ifconnectors are drawn for each shipment, there could be a very largenumber of connectors, including many connectors with the same sourcefacility and destination facility. It may be better to group togetherthe shipments with the same source facility, destination facility, anditem 494, so the user 100 specifies the level of detail to include the“from” facility 488, the “to” facility 490, and the item 494. In someimplementations, there is a connector level of detail shelf, and therelationship based properties can be dropped onto the shelf as describedabove with respect to the size and color encodings 546 and 556 (seeFIGS. 5B and 5C). Having specified the level of detail, the connectorsare aggregated, so the available set of properties for connectorencoding changes. For example, implementations that use a popup windowfor the encodings may use a window as illustrated in FIG. 5D to displaythe available properties. Because the “from,” “to,” and “item”properties are used in the grouping, they are in the set of aggregatedrelationship properties 508′. They can be used for the other encodings.In addition to these three fields, the set of aggregated relationshipproperties 508′ includes five aggregate properties: the number ofrecords, the sum and average of the amount, and the sum and average ofthe cost.

FIG. 6A illustrates an overall process flow for a data visualizationapplication 222 or 322 in accordance with some implementations. Thedatabase 106 includes data fields 238 from data sources 236 andrelationships 240 (identified by the relationship identification module230 or constructed using the relationship builder module 232). In someimplementations, the database 106 also stores other information, such asa history log 244 or user preferences 242. The database 106 may beaccessed over a network 108 or stored locally on a computing device 102of the user 100. After the user 100 selects the data source(s) 236, thecorresponding data fields 238 and relations 240 are loaded (602) intothe schema information region 510 of the user interface 500.

The user 100 uses the user interface 500 to select (604) a set of datafields 238 and to specify how those data fields are used. In addition,the user may select (604) one or more relationships 240 (e.g., asconnectors). The data fields 238 and relationships 240 may specify theoverall layout (e.g., the rows and columns of a Cartesian layout), mayspecify how visual marks or connectors are encoded (e.g., size andcolor), or may specify the level of detail for the desired datavisualization (i.e., at what level the data is aggregated). This wasdescribed above with respect to FIGS. 5A-5D.

The user may also select or specify one or more filters, which aredescribed in more detail below. Filters can apply to visual marks orconnectors between marks, and may apply to individual records from adata source or to aggregated tuples.

As explained above in FIGS. 5A-5D, the data fields 238 are selected fromthe data sources 236. In some implementations, some of the data fields238 are defined according to expressions that combine and/or performoperations on one or more data fields 238 or relationships 240.

After the user has specified (604) the parameters for the desired datavisualization, the data visualization application 222 (or 322) generatesa plurality of queries to retrieve corresponding data and relationshipsfrom the database 106. In some implementations, the queries include nodequeries that retrieve tuples including respective data fields 238. Insome implementations, the queries include one or more link queries thatretrieve relationship information related to the retrieved tuples. Forexample, a relationship 240 between tuples may be defined by having afirst data field of a first tuple equal to a second data field of asecond tuple. See, e.g., first order relationship 410 in FIG. 4B andcorresponding examples in FIGS. 4C and 4D.

In response to the plurality of node and link queries, a set of datatuples are retrieved (606) from the data source(s) 236, and each datatuple includes an ordered set of data fields 238. For each node query,the retrieved tuples have the same structure, including number of datafields, order of the data fields, data types of the data fields, anddata field names. In some instances, there is a single node query. Insome implementations, the relationships among the retrieved data tuplesare included in the retrieved tuple data (e.g., for relationships thatcompare two distinct data fields of two data tuples or for equivalencerelationships). In some instances, the data visualization application222 (or data retrieval module 226) retrieves (606) relationship data aswell. In some implementations, generating the queries and retrieving thecorresponding data is performed (606) by the data retrieval module 226.

As explained in more detail with respect to FIGS. 5A and 7E, the tupledata and relationship data may be intertwined from a calculationperspective. For example, relationships may be constructed from tuples(see, e.g., FIG. 4B-4D), and data fields may be constructed fromrelationships (see, e.g., FIG. 7E). In many cases, a user 100 can usedata fields and relationships in similar ways.

After the selected data is retrieved (606) and processed (e.g.,aggregated or filtered), the data visualization generation module 228generates (606) a data visualization that uses the retrieved data tuplesand relationships to build the desired data visualization. The userinterface module 224 then displays (606) the data visualization on adisplay device 208. In some instances, data tuples are visualized astwo-dimensional or three-dimensional diagrams or charts withoutillustrating data relationships among the data tuples. In someinstances, data tuples are visualized with edge marks (connectors)connecting the data marks representing the data tuples. In someimplementations, the positions of the marks corresponding to the datatuples are adjusted based on relationships between the tuples (e.g.,using mark positions to illustrate a relationship, rather than using aconnector to illustrate the relationship).

As shown in FIG. 6A, the computing device 102 displays a graphical userinterface 500 to support the above process of building datavisualizations. This was described above with respect to FIG. 5A-5D.Included in the user interface is a graphic display region 530. FIG. 6Aillustrates two alternative data visualizations 540A and 540B that maybe displayed in the graphic display region 530. The data visualization540A illustrates a data visualization with 16 individual panes (a 4×4array), where each pane is a “small multiple” and there are noconnectors. The data visualization 540B illustrates a network diagram ofcollege football teams, with sub-groupings by conference. The connectorsindicate games played between teams.

FIG. 6B illustrates other aspects of the process flow for building datavisualizations. At a high level, a user selects (620) one or more datasources, then interacts (622) with a graphical user interface 500 tospecify the properties of a desired data visualization. In someimplementations, the user interactions build a visual specification 234,which is subsequently used to generate and display (646) the desireddata visualization. Once a data visualization is displayed, there arecertain post-generation options 650 that the user can select. In manycases, based on the generated data visualization, the user iterates 660the process, going back to select or update one or more options thatwere previously selected. The ease of making changes and generating anew data visualization facilitates the exploratory process used byanalysts to discover characteristics or patterns in their data.

Based on the data fields 238 and relationships 240 corresponding to theselected data source(s) 236, a user can build (624) additionalrelationships 240 using the relationship builder module 232. Someexamples are provided above with respect to FIGS. 4B-4D. In someimplementations, a user interface for the relationship builder module232 is provided in a popup window, and activated by a button or menuitem in the user interface 500. When a new relationship is constructed,it is typically stored for future use, which may be with the data source236 or in a separate location.

In some implementations, data visualizations are classified according tohow they present data to the user. In some implementations, theclassifications are referred to as “view types” or “chart types.” Insome implementations, the view types are text tables, highlight tables,heat maps, bar charts, scatter plots, line charts, area charts, circleplots, treemaps, maps, pie charts, bubble charts, Gantt charts, boxplots, and bullet graphs. Some implementations include more or fewerview types. In some implementations, some of the view types include twoor more variations or sub-types, so after selection of a view type, theuser is prompted to select an appropriate sub-type as well. A user canselect (626) or change the view type at any time. In particular, FIG. 6Billustrates that the view type can be changed after other features areselected, or even after a data visualization has been generated anddisplayed. This allows a user to quickly view the same data inalternative ways, such as a bar chart or a line chart of the same data.

Some of the encodings for visual marks are illustrated in FIG. 5A,including text encoding 542, color encoding 552, size encoding 544, andshape encoding 554. At any time, the user can select (628) or change themark encodings. For example, a user may initially select color encodingfor the points in a scatter plot, then discover there are too manycolors. If the encoding is based on a quantitative field, the user maychange to using a size encoding.

In many cases, a user filters the data in order to focus on a certainaspect. For example, if a sales database includes data for the past 20years, the user may filter the data so that only the data for thecurrent year is displayed. This is an example of a filter that may beapplied as the data is retrieved from the data source (e.g., retrieveonly records for the year 2014). On the other hand, some filters areapplied at an aggregate level. For example, a user may wish to reviewsales data for all products with sales volume greater than a certainamount (or just the opposite, those products with low sales volumes).This type of filter applies to aggregate data rather than to individualrecords from the data source 236.

In some implementations, distinct user interface tools or windows areused to build/select the two types of filters just described: a tool 630for building or selecting a data source filter and a tool 634 forbuilding or selecting an aggregate filter. In some implementations, anaggregate filter can only be selected if the user has specifiedaggregation of data (e.g., specifying a level of detail). In someimplementations, both types of filters are provided using a singleinterface tool or window, with various configuration options to specifythe type of filter.

Some filter expressions are as simple as comparing a data field 238 to avalue (e.g., sales region=“Central”), but other filter expressions usecomplex expressions that can include multiple data fields,relationships, and/or multiple operators (e.g., arithmetic operators orlogical operators). Relationships can be used to filter the set oftuples that are displayed as visual marks. In some cases, a specifictuple is designated as a root, and the displayed tuples are filtered toinclude only those that are connected to the root tuple by a sequence ofpairs of related tuples. For example, suppose tuple A is the root tuple,A is related to B, and B is related to C. Then tuples A, B, and C wouldall be included in the displayed data visualization. However, if thereis no sequence of related pairs of tuples from tuple A to tuple D, thentuple D is excluded. This example illustrates that relationships can beused to filter the displayed tuples, even when the relationship itselfis not displayed (the data visualization is not required to display anyconnectors).

Filtering can be applied that limits the displayed visual marks or thatlimits the displayed connectors (or both). The examples above havefocused on the displayed visual marks, but filters may be built andapplied to connectors as well. Like visual marks, connectors can befiltered from the source data or from aggregated data. For example,using the data from shipments illustrated in FIG. 4D, shipments can befiltered based on the item transported (e.g., only mocha), filtered by adate range for the shipments (e.g., only shipments in May of 2015), orfiltered by carrier (e.g., only ABC Railway). Only the filteredshipments would be displayed as connectors in the data visualization.Using the same data from FIG. 4D, the transactions could be aggregated(e.g., using the “from” field 488, the “to” field 490, and the “item”field 494), and then filtered (e.g., based on the number of shipments,the total amount, or the total cost). In some implementations, the userinterface tools for filtering connectors are different from thefiltering tools used for the visual marks.

As illustrated in FIGS. 5A and 7E, a user can build (632) additionaldata fields 238 from existing data fields 238 or relationships.Typically the new data fields 238 are stored for later use, and may bestored with the original data source 236 or elsewhere. In someimplementations, a constructed data field comprises a formula ordefinition, and that formula or definition is applied only as needed(e.g., when a data visualization is generated and displayed). In someimplementations, when the data source is a relational database (e.g.,SQL), the formula is translated into a stored procedure or view that isstored with the data source. In some implementations, the formula ordefinition is included in a visual specification 234. In someimplementations, the data values for a constructed data field arecomputed when the formula is saved. For example, if the data source is aspreadsheet or CSV file, some implementations enable the user to savethe computed values of the new field as an additional column in thespreadsheet or CSV file. In general, when a user builds (632) a new datafield, the user can assign a name to the new field. In someimplementations, the same portion of the user interface that enables auser to construct a new field enables a user to assign an alias to anexisting field.

Some implementations support various layout types 714. For example, someimplementations generate and display data visualizations that are“Cartesian,” “Circular,” “Packed,” or “Network.” The layout types 714are described in more detail below with respect to FIGS. 7B and 7C. Insome implementations, the layout type 714 may be selected (636) using amenu item, a toolbar icon, or a user interface control. The selectedlayout type 714 controls what type of data visualization the datavisualization generation module 228 generates and displays. For example,as illustrated in FIG. 7B below in boxes 726 and 728, a user can switchfrom a Cartesian bar graph to a polar bar chart simply by changing thelayout type to Circular. The user can just as easily switch back. Insome implementations, when the layout type 714 is changed, some of thelabels in the user interface 500 change to reflect the updated roles ofthe fields. For example, “rows” and “columns” may become “radius” and“angle” when switching from Cartesian to Circular.

As illustrated above with respect to FIG. 5A, a user can select (638)elements to be used as connectors. In particular, the user can place adata field 238 or a relationship 240 onto the connector shelf 536, whichis then used to specify the connectors to display. Some implementationsallow multiple items on the connector shelf 536.

As illustrated in FIG. 5A, a user can select (648) various encodings forthe connectors 560 using connector encoding shelves 546 and 556. Someimplementations include additional or different connector encodingshelves. The connector encodings specify various characteristics for theconnectors, such as size, color, tapering, shape (e.g., straight linesor arcs), or associated text.

In some implementations, a user can specify how to aggregate the tupledata. In some implementations, aggregation is based on a user selection(640) of the level of detail. Some implementations include a level ofdetail shelf, and the user may place one or more data fields 238 on thelevel of detail shelf. In some implementations, a relationship 240 maybe placed on the level of detail shelf. For example, a first orderrelationship 410 may be treated like the data field on the left side(source) of the relationship. An equivalence relationship 416 may betreated like the data field that forms the relationship. Although a usercould just select a data field 238 instead of a relationship, allowingrelationships 240 provides greater flexibility for users.

When data is retrieved for a data visualization, tuples that havematching values for all of the fields in the level of detail are groupedtogether. The behavior is similar to an SQL “group by” clause. In someinstances (such as an SQL database), the grouping is performed at thedata source 236. In other instances, the tuples are retrieved from thedata source and aggregated at the computing device 102.

Using the level of detail shelf to specify grouping is common fortransactional data. For example, if a sales database includes millionsof individual transactions, grouping may provide more useful informationto an analyst. For example, sales transactions may be aggregated (i.e.,grouped) by customer. Some implementations allow grouping by computeddata fields as well, such as a “quarter” data field computed based on atransaction date. For example, a user may place the two data fields“customer_id” (a field defined in the data source) and “quarter” (afield computed from the “sales_date” field in the data source) on thelevel of detail shelf in order to see data for each customer by quarter.In some instances, when there is grouping by a computed data field, thegrouping is performed at the computing device 102.

Some implementations include a separate connector level of detail shelf.In some instances, the data source or the selected level of detail forthe tuples imposes inherent limits on what may be selected for theconnector level of detail. As illustrated with respect to the example inFIG. 4D, a user may select aggregation for the connectors when there isno aggregation of the tuples. Conversely, even when tuple data isaggregated, the connectors need not be aggregated. For example, supposethe data in FIG. 4D were expanded to include many more facilities,including some cities with multiple facilities. A user could specify“city” as the level of detail for the tuples, but not specify anygrouping for the shipment transactions. Each distinct shipmenttransaction would appear as a separate connector. In this scenario, ashipment between facilities in a single city would appear as a loop atthe visual mark for that city. A user may also filter out shipmenttransactions that are within a single city.

FIG. 5A also illustrates that a user may select (644) elements tospecify the rows and columns for the visual marks. The user accomplishesthis by placing data fields 238 or relationship 240 onto the rows shelf532 or the columns shelf 534. In some instances, when multiple datafields 238 or relationships 240 are placed on the rows shelf 532 orcolumns shelf 534, the outer fields split the displayed datavisualization into panes, such as the data visualization 540A in FIG.6A.

In some implementations, a user can request (642) data visualizationrecommendations based on the current selections. The data visualizationapplication 222 or 322 may recommend a layout type 714, a view type,potentially useful encodings, and so on. In some implementations, therecommendations are based on the data types of the selected data fields(e.g., are they ordinal or quantitative?). In some implementations, therecommendations are based on the data values of the selected datafields. In some implementations, the recommendations are based onrelationships that have been defined or selected.

The recommendation module applies some general rules in order to providegood recommendations. For example, when a relationship is chosen, itshould generally be placed on the connector shelf 536 rather than therows or columns shelves. When a tree relationship is chosen, and thereis a low cardinality, a hierarchy chart is an effective datavisualization (see FIG. 8F). When there is a tree relationship with alarge cardinality, a hyperbolic tree is an effective data visualization(see FIG. 8C). For tree relationships with intermediate cardinality, aforce-directed graph is an effective data visualization (see FIG. 8D).When the relationship is a tree, and a quantitative field is being used,a treemap or hierarchy chart can be effective.

At any point, the user can generate (646) and display (646) a datavisualization corresponding to the current user selections. As describedabove with respect to FIG. 6A, the selected data and relationships areretrieved from the data source, processed as needed (e.g., aggregationand post aggregation filtering), then displayed as a data visualization540 (such as data visualizations 540A or 540B).

Once a data visualization has been displayed, there are variouspost-generation options 650 that a user may select. Of course the usercan view (652) the data visualization, which is a primary objective ofdata visualizations. In some instances, the user will present the datavisualization to others, and thus one of the post generation options 650is to save or export (656) the displayed data visualization. In someimplementations, the data visualization may be saved as a PNG file, aPDF file, a JPEG file, a BMP file, or other standard formats for visualdata. In some implementations, a user may choose to save the underlyingdata with the saved data visualization. In addition, a user may alsochoose to save or export the visual specification 234 for the datavisualization. In some implementations, the visual specification may bestored with the data visualization, or it may be stored separately.Because the data in the data source may change over time, the samevisual specification may result in different data visualizations whenrun at different times. Some implementations, permit saving of a visualspecification during the user interaction phase 622 (i.e., thecorresponding data visualization need not be displayed or even generatedyet).

In addition to saving or exporting a visual specification 234 (whichspecifies how the data will be generated and displayed) or saving a datavisualization 540 (the output generated by the visual specification),some implementations allow exporting the data used by the datavisualization. This may be particularly useful when the data isprocessed in various ways after retrieval from the data source. Forexample, there may be additional data fields that are derived orcomputed, the data may be aggregated or filtered, or the data may besorted. When there is connector data, implementations typically exportthe data for the visual marks separately from the data for theconnectors. In some implementations, the data can be exported to CSVfiles or spreadsheets. Some implementations allow exporting to otherformats.

Some implementations allow a user to manually adjust (654) a datavisualization after it is generated and displayed. For example, a usermay adjust the location of visual marks (e.g., in a network layout). Insome implementations, a user can adjust (654) the location or shapes ofconnectors in a data visualization (e.g., creating arcs rather thanstraight lines, or adjusting locations so that connectors or marks donot overlap). In some implementations, the adjustments are stored aspart of the visual specification 234 so that the adjustments can bereapplied (if possible) if the user makes other selections. In someimplementations, the adjustments are stored in a visual stylesheet,which is separate from the visual specification. In someimplementations, the visual stylesheet stores any manual adjustmentsmade after a data visualization has been generated. For example, a usermay make adjustments to the location of connectors, then decide tochange the color encoding of the connectors. When regenerated, theconnectors are in the same adjusted locations, but use the newlyselected color scheme. In some implementations, manual adjustments arestored with the generated data visualization in addition to or insteadof the visual specification 234. For example, the manual adjustmentsresult in modifications to a generated graphics file (e.g., TIFF, JPEG,or PNG file).

Further examples of post-generation interactions with a displayed datavisualization are provided in additional figures below.

Some implementations provide various zoom in/zoom out features 658. Inaddition to zoom features that behave like ordinary magnification, someimplementations adjust the details displayed based on the magnification.For example, FIGS. 13B-12D illustrate college football teams in theUnited States and games played between them. In FIG. 13B, all of theteams and connectors are displayed, making the display appear somewhatlike spaghetti. However, FIGS. 13C and 13D illustratemagnification-based detail. The high level view in FIG. 13C illustrateseach of the divisions, with games played between teams from differentdivisions. The magnified view in FIG. 13D shows only a single divisionand the games between teams in that division, without the clutter ofgames played against other teams.

In some instances, after viewing, saving, or adjusting a datavisualization, the user is done 662, and closes the application 222 or322. Commonly, however, based on the data visualization, the useridentifies one or more aspects of the data visualization to change. Inthis case, the user iterates (660) the process, interacting (622) withthe user interface 500 as described above. The iterative process may berepeated any number of times. In some implementations, the most recentlygenerated data visualization 540 remains displayed in the graphicdisplay region 530 of the user interface 500 until the user generates anew data visualization.

In some instances, the data source(s) are changed or modified. If theuser starts from scratch with a new data source 236, implementationstypically remove the previous selections (e.g., when the data source isremoved, the data fields on the rows shelf 532 or the columns shelf 534are no longer meaningful. so they are cleared). In some implementations(not depicted in FIG. 6B), a user may add another data source that isblended with an existing data source without losing the user's selectedoptions (e.g., retaining the information in the visual specification).

FIG. 7A illustrates how ordinal and quantitative data fields aredisplayed differently in data visualizations 540. The classification ofdata fields 238 as “ordinal” or “quantitative” can be useful todetermine how data is displayed. In general, a “quantitative” data fieldhas data values that vary over a continuous range of numeric values. Onthe other hand, an “ordinal” data field has discrete values.

The profit data field illustrated in FIG. 7A is quantitative, and spansa continuous range 702. In this particular example, the profit may fallanywhere between 0 and 22 million. In some instances the profit could benegative, so the range 702 would have to account for the negative valuesas well. The range 702 is shown horizontally in FIG. 7A, whichcorresponds to placing the profit data field 238 on the columns shelf534 in the user interface 500 (the columns shelf 534 specifies thex-position of the graphical marks).

FIG. 7A also illustrates two ordinal data fields “quarter” and “region.”For the quarters, assume the data has been filtered to a single year.The ordinal field “quarter” partitions the axis into four discreteportions. The quarter data field is commonly computed based on anunderlying data field with date or date/time values, but some datasources 236 store data for quarters directly (e.g., in a spreadsheetwhere a user has already performed some calculations). If the quarterdata field is placed on the columns shelf 534 in the user interface 500,the quarters are spread out horizontally, such as quarters 704. All ofthe data for an individual quarter (such as Qtr4 706) is groupedtogether. If the quarter data field is placed on the rows shelf 532, thequarters are spread out vertically, such as quarters 708. The fourthquarter Qtr4 710 is displayed as the lowest row.

In another example, if a “region” data field is placed on the columnsshelf 534, the axis 712 is displayed horizontally, with each distinctregion forming a column. For example, region “Southeast US” 714 andregion “Europe” 716 each create a column for data in the datavisualization.

For ordinal data fields 238, some implementations enable a user tospecify the order of the created rows or columns. For example, a user100 may be able to rearrange the five regions in the region axis 712.

The axes displayed in FIG. 7A are horizontal or vertical as used in aCartesian layout. However, the same structure for ordinal andquantitative fields applies to other layouts as well, such as circularlayouts as depicted in FIG. 7B. Sometimes circular layouts are referredto as “radial” or “polar.”

The table in FIG. 7B illustrates sample data visualizations that may begenerated and displayed based on the classifications of the data fieldsspecified for the columns shelf 534 and the rows shelf 532 (whichtogether specify the pane type 716) as well as the layout type 714.There are four distinct pane types (OO, OQ, QO, and QQ), which specifythe data type used for the x-position and y-position of the marks (asspecified on the Columns shelf 534 and Rows shelf 532). Note that thepane type 716 is based on the classification of the innermost datafields in the columns shelf 534 and rows shelf 532. In someimplementations, when two or more data fields are specified for the rowsor columns, the outermost data fields subdivide the data visualizationregion 530 into a plurality of panes, as illustrated by the datavisualization 540A in FIG. 6A.

As described in more detail below, additional types of datavisualizations are possible based on these layout types 714 and panetypes 716. Also, some implementations support additional layout types714, including packed and network, some of which are illustrated belowwith respect to FIG. 7C.

The box 720 illustrates a heatmap data visualization that may begenerated and displayed in a Cartesian layout when both axes use ordinaldata fields (an OO pane type). The heatmap data visualization in box 720is a grid, and each element of the grid is colored based on the colorencoding of some data field (e.g., by placing some data field 238 on thecolor encoding shelf 552 in the user interface 500). Note that othertypes of data visualizations are also appropriate for a Cartesian layoutwith ordinal values used for both the rows and columns. For example, atext table would be a common option. In a text table, rather than acolored rectangle in each grid position, there would be text, whichcould represent the data for another field (e.g., revenue).

The box 722 illustrates a data visualization that may be generated anddisplayed in a Cartesian layout when the x-position (Columns) uses anordinal field and the y-position (Rows) uses a quantitative field (an OQpane type). One such data visualization is a bar chart with verticalbars. Each bar corresponds to a distinct ordinal value and the height ofeach bar corresponds to the value of the quantitative field.

The box 724 illustrates a data visualization with a radial bar chart,which is an appropriate data visualization for an OQ pane type and acircular layout. Each ordinal value corresponds to a sector of the barchart. Generally, each radial bar has the same central angle (e.g., 30degrees) as illustrated in this example. In some implementations, thecentral angle is determined based on the number of distinct ordinalvalues. The sector radius for each bar is determined by the quantitativefield. In the illustrated example, the radial bars are stacked. A usercan easily switch from a Cartesian layout to a Circular layout using thegraphical user interface 500. In some implementations, switching layouttype 714 uses a menu item or toolbar icon. In some implementations,there is a user interface control to select the layout type (not shownin FIG. 5A).

The box 726 is similar to the box 722, but the ordinal and quantitativefields have been reversed. In this case, a bar chart is still anappropriate option, but the bars are horizontal. Each bar corresponds toa distinct ordinal value, and the length of each bar corresponds to theselected quantitative field. In some instances, the displayed lengths ofthe bars are scaled to use the full display space.

The box 728 illustrates a data visualization that may be generated anddisplayed in a circular layout with an ordinal field specified for the“Columns” 534 and a quantitative field specified for the “Rows” 532. Insome implementations, when a radial layout is selected, the labels“rows” and “columns” in the user interface 500 are replaced withalternative labels, such as “Radius” and “Angle.” Here, the ordinalfield corresponds to the radial distance, so bars are created atintervals away from the center. The quantitative field corresponds tothe angle, so larger values are displayed as longer bars wrapping aroundthe circle. In some implementations (as illustrated by the datavisualization in the box 728), the bars always start from a verticalline going upward from the center and proceed clockwise around thecircle. In other implementations, the starting location of the bars isdifferent (e.g., from a horizontal line) or have an opposite direction(e.g., counterclockwise). In some implementations, the starting locationof the bars or the direction of the bars is configurable by the user.

The box 730 illustrates a scatter plot data visualization that may begenerated and displayed in a Cartesian layout with quantitative datafields 238 selected for both the rows 532 and the columns 534. Each pairof quantitative values specifies the location of a corresponding mark inthe scatter plot. Encodings, such as text 542, color 552, size 544, orshape 554 may be used to specify how the marks are displayed. In aCartesian layout, the roles of the two quantitative fields in a QQ panetype 716 are symmetric. Switching the roles of the two quantitativefields mirrors the plot across a 45 degree angle line.

The box 732 illustrates a polar plot data visualization, which isappropriate for a QQ pane type with a Circular layout type 714. For aCircular layout, the “Rows” selection 532 and “Columns” selection 534correspond to radius and angle, and in some implementations, the labelson the display are updated when a Circular layout type is selected. Insome implementations, the values of the quantitative fields are used aspolar coordinates to specify the location for each mark. Using polarcoordinates, an angle of zero corresponds to the positive horizontalaxis 734, and positive values correspond to angles measuredcounterclockwise from the axis 734. Generally, a polar plot isappropriate only when one of the quantitative variables to be displayedrepresents measured angles.

In addition to the Cartesian and Circular layout types 714 illustratedin FIG. 7B, some implementations support Packed and Network layout types714, as illustrated in FIG. 7C.

The box 740 illustrates three types of data visualizations that may begenerated and displayed for OO panes with a packed layout type. Datavisualization 740A is a packed bubble chart in which each mark is acircle or bubble. In general, the bubbles are packed together closely.In some implementations, the size, color, shape, or text of each bubbleis encoded according to user selection in the user interface 500. Insome implementations, related bubbles are grouped together, asillustrated in data visualization 740A. For example, a group of tuplesmay share the same first element and differ only in the second element.The bubbles for these tuples may be grouped together as a bunch orgrouped together as a string.

The data visualization 740B is sometimes referred to as a tag cloud, aword cloud, or a text cloud, and packs together words from some source.For examples, the words may be taken from a document, article, orspeech. In many cases the words are encoded using size or color,indicating the frequency of each word in the source. In some cases, theposition or orientation (e.g., horizontal or vertical) of words is basedon an underlying quantitative or ordinal field.

The data visualization 740C is a treemap that displays hierarchicaldata. In some implementations, the nested structure shown in a treemapoverrides the standard grid structure of panes when a user selectsmultiple data fields 238 for the rows 532 or columns 534. In a treemap,the individual rectangles do not generally align as a two-dimensionalarray.

The box 742 illustrates a data visualization with a Network layout type714 and OO panes. Node-link diagrams, such as the one illustrated in box742, typically include edges in addition to the node marks, as describedthroughout this disclosure.

Boxes 744 and 748 indicate that the data visualizations in box 740 canbe adapted in certain ways when one of the data fields is quantitativerather than ordinal. In particular, a quantitative data field mayintroduce additional “forces” that affect the placement of nodes. Forexample, if the data field selected for columns 534 (the x-position) isquantitative, those quantitative values may be interpreted as forcespushing nodes horizontally to the right. Nodes with larger quantitativevalues are pushed further to the right.

The box 752 illustrates a data visualization with a packed layout whenquantitative fields are used for both the rows and columns. In thisexample, the quantitative variables are the longitude and latitude ofstates in the United States, with sizes of marks encoded according topopulation and color of marks encoding obesity rates. Each of thecircles is in approximately the right location geographically, but thereare some adjustments in order to accommodate the sizes of the circlesand remain packed. This type of data visualization is sometimes referredto as a Dorling cartogram.

As indicated in boxes 746, 750, and 754, networked layouts that have atleast one quantitative field for rows 532 or columns 534 generatenetwork diagrams that are force-directed or constraint based, andinclude additional forces based on the underlying coordinate system. Insome implementations, the labels “rows” and “columns” in the userinterface 500 are modified for network layouts to indicate how the datafields are used.

FIGS. 7B and 7C illustrate various ways to visualize data usingdifferent layout types. FIG. 7D focuses on Cartesian layouts, butexpands the set of options for pane type 716. In addition to selectingpairs of fields that are ordinal (O) or quantitative (Q), datavisualizations can be created when no data field is selected for one ofthe shelves (depicted as a “−” in FIG. 7D). In addition, FIG. 7Dillustrates the case where a relationship 240 (R) is placed onto therows shelf 532 or the columns shelf 534. The example data visualizationsin FIG. 7D have nothing selected for the connector shelf 536, and thusthe chart is labeled xy-(756). Adding connectors is illustrated in FIG.8A. The lower labels 776 indicate what type of data field is selected onthe columns shelf 534, which identifies the x-position of the visualmarks. The side labels 774 indicate what type of data field is selectedon the rows shelf 532, which specifies the y-position of the visualmarks.

The box 758 illustrates the case where no data fields 238 have beenselected for either the rows or columns. Having selected nothing, thereis no data visualization.

The box 760 represents pane type -O, with nothing selected for columns(no x-position), and an ordinal field for rows, specifying they-position. In this case, the retrieved tuples may be displayed as alist (e.g., with text encoding). In some implementations, the elementsof the list may be sorted using another encoding (e.g., alphabeticallyor numerically based on the displayed data).

The box 764 is similar to box 760, but uses a quantitative field 238 forthe y-position. In some cases, this creates a data visualization that isa distribution of the quantitative values (e.g., a line or dot for eachvalue next to a vertical scale). In some implementations, thequantitative values may be grouped together (either by having exactlythe same value or split into intervals), with a visual mark indicatingthe number of instances for each value. Some implementations allow auser to specify this using a level of detail shelf.

The box 766 presents a data visualization that may be displayed wherethere is no specification of x-position, and a relationship 240 is usedto specify the y-position (i.e., a “-R” pane). In some implementations,when the relationship 240 is a first-order relationship 410 (or secondorder 412 or higher 414), using the relationship 240 in the rows shelf532 or columns shelf 534 is almost equivalent to selecting the datafield used as the source of the relationship 240. For example, using therelationship {ID=father_id} 240 (described above with respect to FIGS.4B and 4C) is essentially the same as using the ordinal field ID. Butusing a relationship 240 has some benefits. First, the user can specifyorderings that would be difficult using the corresponding ordinal field.The second is that a relationship can imply more than a simple1-dimensional ordering, allowing a list of values to be indented basedon relationship information, making for richer labels. This isillustrated by the data visualization in box 766, where the indentationis based on the depth of the relationship. For example, continuing withthe {ID=father_id} example, the oldest generation could be positionedthe furthest to the left, with younger generations further and furtherto the right.

For an OO pane as in the box 762, one appropriate data visualization isa text table as illustrated. Another option is a heatmap grid, asillustrated above in box 720 of FIG. 7B. Boxes 722 and 730 weredescribed above in FIG. 7B, but are included here for completeness.

As noted above for box 766, a relationship 240 can sometimes be treatedas an ordinal field with some added benefits. Because of that, the datavisualizations in boxes 768 and 770 look much like the datavisualizations illustrated above in boxes 720 and 726 in FIG. 7B.However, on the vertical axis (corresponding to the relationshipselection for rows), the labels are indented according to therelationship. In this way, a single data visualization is able to conveyeven more information.

The box 772 illustrates a data visualization that may be generated anddisplayed when relationships are used for both the x-position and they-position. Using underlying ordinal fields 238 corresponding to each ofthe relationships 240, the main data visualization may be a text tableas illustrated or a heatmap grid as illustrated in box 720 in FIG. 7B(or other data visualization types). Here, because relationships areused for both the rows and the columns, the axis labels on both axes areindented according to the relationships.

The data visualization examples in FIG. 7D where a relationship is usedillustrate the case where the relationship is a first order relationship410 (or second order 412 or higher 414). As illustrated in FIG. 4B,there are additional relationship types, such as an equivalencerelationship R_(E) 416. For an equivalence relationship 416, when therelationship is used on the rows shelf 532 or the columns shelf 534, itbehaves essentially like the data field 238 that is used as the basis ofthe equivalence relationship, which may be ordinal or quantitative. Anadditional advantage of using an equivalence relationship 416 for rowsor columns is that an automated tool for identifying good datavisualizations has more information, and thus may be able to make betterrecommendations. Because an equivalence relationship has no impliedordering, using an equivalence relationship for rows or columns wouldnot have the indentation illustrated above in boxes 766, 768, 770, and772.

More generally, a relationship 240 can be used to define a new ordinalfield 238 when the relationship creates a sort order of the underlyingdata. For example, if a relationship creates a tree (R_(T) 426), then adepth-first traversal or breadth-first traversal of the tree creates anordering. If the tree consists of multiple portions that are notconnected to each other, then the traversal has to traverse each of theconnected portions, and the order of traversing the groups may bearbitrary. The new ordinal field defined by the sort is effectively thevalues “1,” “2,” . . . , where each of these ordinal values correspondsto a unique tuple. Generally, when the relationship includes loops(e.g., A relates to B, B relates to C, and C relates to A), thetraversal avoids processing a tuple A second or subsequent times.Defining a new ordinal field 238 in this way using a relationship 240may be done independently of any specific data visualization, and storedin the database 106. In this way, the defined data field 238 appears inthe schema information region 510 in the user interface 500.Alternatively, the new ordinal field 238 may be defined as part ofplacing the relationship 240 onto the rows shelf 532 or the columnsshelf 534 (e.g., using a popup window so that a user can specify how therelationship 240 will be used).

A relationship can be used to build quantitative fields as well. FIG. 7Eillustrates multiple ways of defining a quantitative field 238 based onan equivalence relationship 416 or a tree relationship 426, but the sametechniques may be applied to other types of relationships as well.

An equivalence relationship 416 partitions the tuples into distinctgroups, which are sometimes referred to as equivalence classes. Atraversal of all the tuples traverses one group at a time, and traverseseach group before going on to the next group. In general, this involvesmultiple arbitrary choices, including the order to traverse the groupsand the order to traverse the tuples within each group. In some cases,the traversal may be directed by the data within the tuples. Forexample, in the family tree data 438 (FIG. 4C), an equivalencerelationship may be defined by people having the same father. Within agroup of siblings, a logical traversal order is based on birth date 454.Following this same example, each group has a unique father, so thetraversing of the groups could be based on the birth date of the fatherfor each group (e.g., if the father of group A was born before thefather of group B, then group A is traversed before group B). When thedata includes fathers with the same birth date, some arbitrary decisionsabout traversal order would still have to be made. Regardless of how thetraversal is performed, it provides an ordering of the tuples, which canbe used to define several quantitative fields.

For each tuple, the value of the quantitative field Q(R_(E), overall)780 is the overall position of the tuple in the traversal justdescribed. Note that these values are unique. As described below, aquantitative field Q(R_(E), overall) 780 can be used to define theplacement of tuples within a data visualization.

For each tuple, the value of the quantitative field Q(R_(E), group) 782specifies the traversal order for the group of which the tuple is amember. As noted above, the traversal processes each group in itsentirety before moving on to the next group, so there is a unique orderto the processing of the groups. If tuple A is a member of the groupthat was the 39th group, then the value of Q(R_(E), group) for thistuple is 39.

For each tuple, the value of the quantitative field Q(R_(E), local) 784is the traversal order of the tuple within its group. For example, if agroup includes three tuples, the values of Q(R_(E), local) for thetuples in the group are 1, 2, and 3. For a singleton group, the value ofQ(R_(E), local) for the one tuple is 1. In some implementations, thisquantitative field is identified as Q(R_(E), within-group)

For a tree relationship R_(T) 426, FIG. 7E illustrates six differentquantitative fields that may be defined once a specific traversal of thetree is selected. Note that a tree is not required to be fully connectedhere. Furthermore, the same methodology can be applied to any graph(e.g., containing cycles). In that case, a traversal of the tuples justavoids traversing a tuple more than once. Some edges may not be used inthe traversal. When a graph (or tree) consists of two or more groups ofnodes that are disconnected from the other groups, the groups aretraversed one at a time, possibly in an arbitrary order if there is nonatural ordering of the groups based on the data in the tuples. Inaddition, the same methodology can be applied whether the graph isdirected or undirected. When the graph is directed, the traversalfollows the directions of the edges, but in the undirected case, thetraversal of edges can go either way. Commonly, within each connectedgroup, the traversal is depth first or breadth first.

For each tuple, the value of the quantitative field Q(R_(T), overall)786 is the overall position of the tuple in the traversal. Similar tothe quantitative field Q(R_(E), overall) 780, the overall positions areunique, and the values can be used to determine placement or othercharacteristics of visual marks.

For each tuple, the value of the quantitative field Q(R_(T), depth) 788is the depth of the tuple in the traversal. For a fully connected tree,the depth is just the distance from the root (i.e., the starting tuple).The depth of the root itself is 0. In a tree with multiple distinctgroups that are disconnected from each other, there is a local root foreach group, and the depth of each tuple is the distance from its localroot. In this case, each local root has a depth of 0. In someimplementations, when there are multiple groups, each local root isassigned a depth of 1, imagining a (non-existent) top level root ofdepth 0 that connects to each of the local roots. Note that in the moregeneral case of a graph, the selected traversal can affect the depth ofa tuple, because there may be multiple alternative paths from a localroot to a tuple, and the alternative paths may have different numbers ofedges.

For each tuple, the value of the quantitative field Q(R_(T), local) 790is the index of the tuple within its group of siblings. For a tree, theconcept of siblings is well known (i.e., all of the tuples that have thesame parent tuple). The traversal imposes a specific order. Note thatthe siblings are not necessarily traversed consecutively (e.g., a depthfirst search traverses the descendants of a tuple before proceeding withthe siblings of the tuple), but the traversal does impose an order. Ifthere is a set of siblings with four members, then the values of thequantitative field Q(R_(T), local) for these siblings are 1, 2, 3, and 4according to the order in which they are traversed. In someimplementations, the local index values start at 0 (e.g., 0, 1, 2, and 3in the previous example). In some implementations, quantitative fieldQ(R_(T), local) 790 is written as Q(R_(T), within-group) orWITHIN-GROUP(R_(T)).

For a graph that is not a tree, a tuple may have multiple parents.However, a traversal effectively builds a tree. After the traversal iscomplete, the sibling concept is well-defined. Therefore, thequantitative field Q(R_(T), local) can be extended to work withrelationships that are not trees.

For each tuple, the value of the quantitative field Q(R_(T),child_count) 792 is the number of direct children of the tuple. The samefield 792 is meaningful for graphs generally after a traversal has beenselected.

For each tuple, the value of the quantitative field Q(R_(T), desc_count)794 is the number of descendants of the tuple, which includes children,grandchildren, and so on. The same field 794 is meaningful for graphsgenerally after a traversal has been selected.

For each tuple, the value of the quantitative field Q(R_(T), desc_depth)796 is the maximum depth of any descendent of the tuple. The same field796 is meaningful for graphs generally after a traversal has beenselected.

One of skill in the art recognizes that additional quantitative fieldsmay be defined based on one or more relationships. For example, thequantitative field Q(R_(E), group) 782 can be extended to apply to anyrelationship that creates a graph.

The data fields 238 corresponding to columns in the data source 236, aswell as data fields 238 that are derived from data fields orrelationships (e.g., the derived quantitative fields illustrated in FIG.7E) are presented in the schema information region 510 of the userinterface 500. When selected, data for the data fields 238 is retrievedor computed, and used as specified to display visual marks. Each datafield 238 or relationship 240 may be used to define other data fields orrelationships.

As described above, various schema elements from the schema informationregion 510 in the user interface 500 may be placed in various otherlocations or shelves to use the elements or build new ones. In someimplementations, the following actions occur based on dragging aspecific schema element to another location in the user interface 500:

-   -   dragging a relationship 240 from the relationships section 506        to the connector shelf 536 adds a new relationship to the visual        specification 234, which will result in displaying connectors        based on the relationship;    -   dragging a relationship 240 from the relationships section 506        to the rows shelf 532 or the columns shelf 534 adds a new        relationship to the visual specification 234, which will result        in the relationship specifying the y-position or x-position of        the marks;    -   dragging a relationship 240 from the relationships section 506        to the measures section 504 builds a quantitative distance        measure on the relationship, such as those identified in FIG.        7E. In some implementations, the user is prompted to select a        specific distance measure to build;    -   dragging a measure or dimension (from the dimensions section 502        or measures section 504) to the relationships section 506        creates a new equivalence relationship based on the measure or        dimension;    -   dragging a measure or dimension to the connector shelf 536        creates a new equivalence relationship based on the measure or        dimension, and adds the new relationship to the visual        specification 234, which will be used for displaying connectors.

FIG. 8A provides some examples of data visualizations that visualizeboth data and relationships among the data. This set of examples issimilar to the examples in FIG. 7D, but these include a relationship Rto specify connectors (e.g., using the connector shelf 536). Because arelationship R is selected for the connector shelf 536, the chart islabeled xyR (804). The lower labels 806 indicate what type of data fieldis selected on the columns shelf 534, which identifies the x-position ofthe visual marks. The side labels 808 indicate what type of data fieldis selected on the rows shelf 532, which specifies the y-position of thevisual marks.

When a user specifies connectors, connector marks are added to thegenerated data visualization. The connector marks are typically referredto herein as “connectors” or “edge marks.” Each of the connectorscouples together visual marks to show the relationship. In some cases,the connectors have a corresponding direction, which may be depictedusing arrows on the connectors.

As illustrated in box 802, sometimes a user does not select data fieldsto specify the x and y coordinates of visual marks. Instead, thelocations of the visual marks corresponding to the tuples may beselected based on the connectors (e.g., to avoid overlap). Commonly, thedata visualization application 222 spreads out the visual marks in orderto make the data visualization as readable as possible. In someimplementations, after a data visualization is generated and displayed,the user can manually adjust the locations of the tuples to create amore aesthetic or customized visualization. In some cases, a graphic asdepicted in box 802 is referred to as a node-link diagram.

In other instances, one or both of the axes is associated with aspecific data field (or a relationship), as illustrated in each of theboxes of FIG. 8A other than box 802. In each case, a portion of the datavisualization displays the tuple data, and in that respect, the graphicsare generally similar to those illustrated in FIG. 7D. In addition tothe tuple data, each data visualization includes edge marks between someof the tuple pairs. In some implementations, the edge marks aredisplayed as straight lines, as illustrated in box 814, where listelements are connected. In some implementations, edge marks may becurved, such as the edge marks in box 812. In some implementations, thetype of edge marks depends on the type of selected characteristics(e.g., layout type 714, pane type 716, or view type).

In some cases, the addition of a relationship for connectors changes thetype of data visualization that is displayed, as illustrated in box 810.As shown in box 770 in FIG. 7D, an appropriate data visualizationwithout connectors is a horizontal bar chart. But when a relationship isadded, an appropriate data visualization may be a Gantt chart. A Ganttchart may use horizontal bars, but the connectors show relationshipsbetween the bars (e.g., precedence).

FIG. 8B illustrates a map graphic 816 that uses geographic coordinatesto superimpose visual marks and connectors on top of a North Americanmap. The three character data type acronym QQR 818 indicates that theuser has selected a quantitative field Q to specify the x position(using the columns shelf 534), a quantitative field Q to specify they-position (using the rows shelf 532), and a relationship R for theconnector shelf 536. The tuples represent cities, and the connectorsrepresent airline flights between the cities. In this case, theplacement of the visual marks is determined by the longitude 820 andlatitude 822 of each visual mark, which the user has selected for thecolumns shelf and the rows shelf. In this example, the relationship hasno effect on the location of the visual marks, because the placement ofthe visual marks is determined by the user selected fields for rows andcolumns. Also, because the locations of the marks are determined bygeographical coordinates here, a user typically cannot adjust (654) thelocations of the visual marks after a data visualization has beengenerated (e.g., a user cannot choose to move Phoenix to anotherlocation).

In some implementations, a map layout such as the one illustrated inFIG. 8B is selected by choosing a map view type. In general, when a userspecifies a map view type, the user must also specify a map to be used(e.g., the name of a PNG or JPEG file that will be used as thebackground map). From the connectors 560 in the map 816, it is easy torecognize airline hubs, such as Phoenix 826. Although depicted in blackand white here, a user could use color encoding to identify thedifferent airlines.

In FIG. 8C, the user has not specified either the x or y coordinates ofthe visual marks, so it can be described by the data type acronym --R830. Since no data fields are specified for determining the position ofthe marks, the data visualization generation module 228 is free to placethe marks in a way that displays the relationships effectively. Inaddition to the automated process of selecting the location, someimplementations further allow the user to drag (654) marks to newlocations (e.g., for aesthetics or to provide an alternative view thatmay display important aspects of the data).

In some implementations, the hyperbolic tree 828 is selected based onthe combination of the data type acronym --R 830 in conjunction with anetwork layout type 714, and a designated view type.

The data used for FIG. 8C is hierarchical, and displayed as a hyperbolictree 828. Classifying a relationship can help determine the best type ofvisualization. For example, some implementations use the classificationsidentified in FIG. 4B (directed or undirected, a tree versus anarbitrary graph, and so on). Some implementations use the cardinality ofthe relationship (i.e., is the relationship 1 to 1, 1 to many, or manyto many). The hyperbolic tree 828 in FIG. 8C uses the fact thatprogressing outward from the center is 1 to many. In some instances, auser can assist in the layout by choosing a root node for a tree (orroot nodes where the tree is not connected). If a sort order has beendefined (e.g., by a traversal), the root notes are implicitly defined.In fact, a sort order for the tuples (e.g., based on a relationship) canbe taken into account in other ways as well, such as how the nodes arearranged in a hierarchy.

FIG. 8D displays another data visualization that has a --R data typeacronym 830. In this case, the data visualization is a social networkgraph 832, which is displayed using a force directed graph layout.Unlike the data in FIG. 8C, the data here is not a tree (it is anarbitrary graph), so the force directed graph layout is appropriate.

FIG. 8E illustrates another data visualization that may be generated anddisplayed when the data type acronym is --R 830. In this case, therelationship defines a tree and the user has selected a size encoding.The user may select a treemap view type to get the tree map datavisualization 834. For a treemap data visualization, color encoding canalso be useful. The example shows postings to a user group during ayear, where nested boxes show the hierarchy of responses, size shows thenumber of postings, and color indicates how much the number of postingsincreased or decreased. Note that in this example, the connectors arenot displayed as edge marks. Instead, the relationship is displayed bythe nesting in the rectangular hierarchies.

FIG. 8F displays a family tree hierarchy 836, which appears in FIG. 5Aabove as the displayed data visualization 540. This example is based onthe sample data in FIG. 4C. As seen in the data in FIG. 4C, Abe 838 isthe father of Bob 840 and Henry, and Bob 840 is the father of Dave 842and Edith. The relationship R is used to define the connectors 560(e.g., by placing the relationship R on the connector shelf 536). Inorder to place the descendents further to the right, a computedquantitative field Q_(R) is placed on the columns shelf 534. (See, forexample, computed field 788 in FIG. 7E.) The computed quantitative fieldQ_(R) uses the relationship to determine distance, which in this casemeasures the distance of a node from the root (the root is Abe 838).

Because the position along the vertical axis is not selected by theuser, the data visualization generation module 228 can arrange the nodes(the boxes) to avoid overlap. The generation module can assigny-coordinates as needed because they have not been selected by the user.

This example also shows multiple data fields used for a text encoding,with the display adjusted for the text content.

FIG. 8G illustrates a graph 846 in which the user has not specified datafields for x or y positions. That is, the data type acronym is --R 830.This may be generated, for example, by selecting a network layout typeand an appropriate view type (e.g., graph). Unlike the graphs in FIGS.8C and 8D, however, the user has chosen a size encoding for the visualmarks (the nodes) so that some of the nodes are larger than others. Inthis instance, rather than continuous sizing, each of the nodes is oneof three sizes.

FIGS. 8H, 8I, and 8J illustrate data visualizations in which connectorsare used, and the user has specified both the rows and columns. Based onthe data types selected for the rows and columns, different types ofdata visualizations are generated.

The family hierarchy 848 in FIG. 8H uses the data from FIG. 4C. In thiscase, the x-axis 852 is associated with birth and death dates of thepeople, the y-axis is based on the father-child relationship, and theconnectors are based on the same father-child relationship. The datatype acronym here is QRR 850. The relationship R selected for the y-axisdefines the vertical order, but the data visualization generation module228 can determine the spacing arrangement.

FIG. 8I illustrates semantic substrates 854, which are based on a datatype acronym QOR 856. The data visualization in FIG. 8I includes aplurality of visual marks that represent related case laws in the U.S.Supreme Court and a circuit court. Here, the x-axis 858 is associatedwith year of each case law, and the y-axis 860 is associated with twocourts (i.e., a “row” for the Supreme Court and a row for the circuitcourt). Visual marks are placed to represent relevant cases based ontheir respective year and court. In some instances, a user may specifyan equivalence relationship 416 for the rows, where the equivalencerelationship is based on both court and subject matter

There are various relationships between the cases based on citation. Inone example, citations are instances where the circuit court cites anearlier Supreme Court case. In another example, the relationship isbased on the Supreme Court taking an appeal from the circuit court. Insome instances, the tuples are aggregated based on court, year, andsubject matter (e.g., using a subject matter classification in therecords from the data source). When the tuples are aggregated, theconnectors are commonly aggregated as well. In some cases, theconnectors have a size encoding based on the aggregated number ofrelationship instances. In some instances, a user applies a filter tothe connectors based on the year of the circuit court case. For example,in FIG. 8I, there are many dots corresponding to cases, but connectorsare displayed only for citations for 1995 circuit court cases. SeeSchneiderman, B., Network Visualization by Semantic Substrates, IEEETransactions on Visualization and Computer Graphics, 12(5), 733-740,2006. Disclosed implementations can build such graphics using the userinterface 500 rather than constructing the graphics manually.

FIG. 8J illustrates a dendrogram 862, which may be used when the datatype acronym is QQ_(R)R 864. The dendrogram 862 is associated with ahierarchical clustering. The y-coordinates (rows shelf 532) use aquantitative field constructed from a relationship, such as depth 788described with respect to FIG. 7E. The x-coordinates (columns shelf 534)use another quantitative field. The visual marks corresponding to thedata tuples may be sorted along the corresponding x-axis of thedendrogram in accordance with specific clustering criteria of thesetuples. The connectors are based on the same relationship used toconstruct the quantitative field used on the y-axis.

FIG. 8K illustrates a chart (870) of the elements that uses an OOR datatype acronym 872. This is called a Hull Periodic Chart. This exampleshows the power on the data visualization application 222 and userinterface 500, even if the application 222 might be used in thisspecific way. The underlying data for the tuples includes data fields238 for each chemical symbol, the period of each symbol (correspondingto the valence shell of electrons), the atomic number of each element,and other data for each element.

In the Hull Period Table 870, the period 874 is placed on the rows shelf532, such that elements in the same period are displayed in the samerow. The period is effectively an ordinal field because of the discretevalues. Using a traversal of the elements by atomic number, a computedfield within-group([Period]) 876 is defined, which computes the order ofthe elements within each period. This is similar to the computed fields784 and 790 in FIG. 7E. Because the computed values are discreteintegers, this computed field is effectively an ordinal field as well.The selections on the rows shelf 532 and columns shelf 534 determine thelocation for each element.

An interesting aspect of this chart 872 is that is uses two distinctrelationships 878 and 880 between elements. These relationships aresometimes referred to as principal and secondary. For example, theelement Hydrogen 882 is connected to Lithium 886 by a connector 884 (theprincipal relationship 878), and connected to Fluorine 890 by a secondconnector 888 corresponding to the secondary relationship 880. In someimplementations, connectors corresponding to the different relationshipsare encoded with different colors. In some implementations, the elementsthemselves are color coded (e.g., to indicate the element is a solid,liquid, or gas at a standardized pressure and temperature). Encodingsmay indicate other properties, such as whether an element is a metal,the density of the element, and so on.

FIG. 8L provides another treemap 892 data visualization, and displays aportion of the user interface 500 that shows how data on the shelves areused to create the treemap data visualization 892. Like the othertreemap 834 in FIG. 8E, the data type acronym is --R 830. The “--” inthe data type acronym 830 corresponds to the fact that no elements havebeen placed on the Rows shelf or the Columns shelf. However, therelationship {Sector} 894 has been placed on the Connectors shelf.

The {Sector} relationship is a category tree hierarchy, similar to theone illustrated in FIG. 4E. At the top of the hierarchy are the labeledsectors “Health Care,” “Financial,” etc., as seen in the displayedtreemap 892. Each sector is subdivided into industries, which form thesmaller rectangles within each sector. In some instances, the industriesare further subdivided into smaller groupings. The {Sector} relationshipcorresponds to the sector/industry hierarchy.

Unlike a typical connector, which is visualized as a line or arc, theconnectors here are visualized by the hierarchy of rectangles: when anindustry is related to a sector in the hierarchy, the rectangle for theindustry is inside the rectangle for the sector. Here, the user hasspecified “Market Cap” 896 for size encoding of the connectors. Usingthe size encoding produces rectangles that are proportional in size tothe market capitalization of each industry. This example includes acolor encoding based on “% Change” 898, which is the percent change inmarket capitalization during a certain period of time. In this way, itis easy to identify the industries that are growing or shrinking.

In some implementations, when a treemap is generated and displayed, auser can zoom in to get more detail about any portion of the treemap.Based on the magnification level, more detail is provided.

FIG. 8M illustrates both filtering and sorting based on a relationship.This figure presents another family tree hierarchy, and uses therelationship {ID=father_id} as described above with respect to FIG. 4C(but uses a different set of data). As shown at the top of FIG. 8M, thecolumns (x-position) 954 are specified by the quantitative functionDEPTH( ) based on the relationship {ID=father_id}. The rows (y-position)952 are specified by the relationship itself, and the connectors 956 usethe same relationship {ID=father_id}. Based on these settings, the datavisualization has data type acronym Q_(R)RR 950.

A filter has been applied to limit the depth of the family tree to fourlevels. Filtering based on depth within a tree relationship is alsodescribed below with respect to FIG. 14 and FIGS. 21A-21C. Note that thedepth is based on a traversal of the tree.

The traversal of the tree also provides a unique order to the nodes (theperson tuples). In some implementations, when a relationship is selectedto specify the rows or columns, the elements are sorted based on thetree traversal, as illustrated here. For example, in the second column958, the children of William Henry Gorman are displayed in the traversedorder. The traversed order is not necessarily tied to any data of thetuples, but the traversal may use tuple data if available. For example,if birth date information is available for all of the people, thetraversal may use that information when deciding which node to traversenext. In a breadth-first traversal, all of William Henry Gorman'schildren (i.e., the people in the second column 958) are traversedbefore other descendents. However, a depth-first traversal, otherdescendents are traversed before all of the children. Regardless of thetraversal algorithm, the result is a unique order for all of theelements in the tree. If a subset of the nodes are selected (e.g., thenodes in the second column 958), there is a unique order of those nodes,which is used as the sort order here.

The fourth column 960 includes great grandchildren of William HenryGorman. With either a breadth-first traversal or a depth-firsttraversal, the great grandchildren are sorted in such a way that thepeople with the same father are sorted together (e.g., the five childrenof William Baker Gorman are sorted together). Note that sorting thegreat grandchildren by their birth dates would not put siblingstogether, creating a data visualization that is either messy, not aseasy to read, or both.

Using a relationship 240 to sort elements in a data visualization canalso be applied to other layout types 714, such as a circular layout.For example, if a relationship 240 is selected to specify angularposition in a circular layout, the placement of the nodes in the layoutis based on the traversal order. In some implementations, sorting can beapplied within designated subsets as well, as illustrated below in FIGS.13A-13D (sorting the teams within each division according to a circularlayout).

FIGS. 9A-9C illustrate generating and displaying data visualizationswith certain aesthetic qualities. How data is presented can be veryimportant in order for end users to understand and retain the presentedinformation. Each of the visual representations 900, 910, and 920includes a plurality of visual marks and connectors in accordance withsome implementations. Each of the plurality of marks represents at leastone tuple retrieved from a data source 236. In some instances, each markis displayed as a simple geometric shape (e.g., circle, square,triangle, and diamond) or an image. In some instances, each mark isrepresented or accompanied by a text label (e.g., using the textencoding shelf 542 in user interface 500). The marks are linked to eachother by the connectors. The geometric shapes, images, text labels,and/or connectors preferably do not overlap each other, or overlap aslittle as possible based on the data to be presented. Having overlap ina data visualization reduces the effectiveness of the visualization bothfunctionally and aesthetically. From a functional standpoint, certaindata is either obscured or confused. And from an aesthetic standpoint,having a “good” graphic keeps users engaged with the presentation.Therefore, a key factor evaluated by the data visualization generationmodule 228 is how well the data is displayed (e.g., readable, usable,etc.).

In some implementations, the locations of the visual marks in a datavisualization are not explicitly or implicitly associated with anordinal or quantitative data field of a retrieved tuple (e.g., the rowsshelf 532 and the columns shelf 534 in the User interface 500 are leftblank). Therefore, the data visualization generation module 228 selectsthe locations of the visual marks The data visualization generationmodule 228 includes a plurality of layout algorithms, and applies anappropriate algorithm based on the visual specification 234 (e.g., thelayout type, the pane type or data type acronym, the view type, and soon).

The data visualization 900 illustrates determining the locations ofvisual marks 902 to accommodate corresponding text labels. The sizes ofthe text labels are encoded according to the sizes of the visual marks902 to prevent the text labels from overlapping with each other. In someinstances, a text label that is semantically associated with a visualmark is too long. In some instances, only a part of the text label isdisplayed with the corresponding visual mark (e.g., using truncation).In some instances, the text label is wrapped for display on multiplelines. In some instances, text labels are displayed that extend outsideof the visual marks. In some instances, text labels are selectively usedfor some, but not all, visual marks because of limited space. The textmarks that are not shown on the data visualization 900 may be displayedwhen a user chooses to zoom in on a particular portion using the zoomfeature 658 (as indicated by the icon 906), or in some implementationswhen a user hovers a cursor at a certain location.

The zoom feature 658 is an interactive post-generation option 650, whichis particularly useful for a graphic such as 900 that is based upon alot of data. Some implementations provide magnification-based detail(such as text labels in graphic 900) as appropriate for the level ofmagnification selected.

In some implementations, when the visual marks 902 are accompanied bytheir text labels, the data visualization generation module 228implements a dynamic label placement method that places the text labelsin accordance with a set of predetermined visual effect criteriaspecifically related to text labels. According to this set of visualeffect criteria related to text labels, the dynamic label placementmethod avoids overlapping labels, makes labels readable, and places asmany labels as possible on the data visualization 900.

In some implementations, when an image is used to represent a visualmark (e.g., digital photos of people in a social network), the imagecharacteristics (e.g., location, size, and resolution) are selected toprovide satisfactory visual effects (e.g., select image sizes so thatthe images do not overlap).

The style of connectors affects the aesthetics and readability of avisual representation. In some implementations, the connector encodingsinclude connector type, arrow location, color, and width. In someimplementations, the connector types include straight connectors 904 andcurved connectors 924. In some implementations, the connectors may haveeither fixed width or tapered width (width gradually changes from oneend of the connector to the other end). Some implementations includearrowhead connectors 908 (which may have arrowheads on either or bothends). Some implementations include arrow connectors where the arrowsare placed in a middle portion of the connector 908. Someimplementations include additional settings for connectors, such as atransparency setting that specifies the transparency level. When used,the transparency level of the connector determines whether visual marksthat overlap with the connector may still be partially visible under theoverlapping areas.

A connector shows a relationship among several data tuples not only bylinking visual marks that represent the data tuples together, but alsoby having visual characteristics that are displayed according to therelationship. For example, some connector encodings (e.g., the linewidth of the connector) may show aggregated or other numeric propertiesof the relationship. This is illustrated in graphic 910, whichrepresents women's responses to survey questions about their personalrelationships. The line width of each connector 908 is proportional tothe number of interviewees that responded in each way. In this case, thenumber of people corresponding to each connector 908 is also displayedas a text mark 912 next to the corresponding connectors 908

Curved connectors provide greater flexibility to display dataeffectively. As shown in the data visualization 910, the curvedconnectors 908 are separated at their common origin (i.e., the textlabel “why doesn't he”), and thus the widths of the connectors aredisplayed with improved visual effects.

As illustrated in data visualization 920, the curved connectors offerimproved visual effects compared with straight lines, especially whenthey are used to connect visual marks that are very close to each other.In some implementations, the curvature of a curved connector is selectedaccording to the visual marks that may overlap the curved connector. Insome implementations, the connectors are curved in order to avoidcrossing the visual marks. In some implementations, certain visual marksor connectors are identified as having a higher priority, so crossingthose marks or connectors is not permitted. Note that the graphic 920uses a circular layout for the marks, with spacing selected so that thevisual marks around the perimeter are roughly equally spaced.

In some implementations, the data visualization generation module 228includes a plurality of dynamic layout algorithms, which arrange thevisual marks and the connectors together according to a set ofpredetermined visual effect criteria. The dynamic layout algorithms usethe visual specification 234, including the information in rows shelf532, the columns shelf 534, the connector shelf 536, the encodings forvisual marks, and the encodings of the connectors, to arrange the marksand connectors. The dynamic layout algorithms also use the tuple dataand connector data to identify visual marks that are potentially locatedon the path of the connectors, as well as the curvature and width of theconnectors themselves. According to this set of visual effect criteria,the dynamic layout algorithms avoid overlapping marks and connectors asmuch as possible, and make connectors discernible. In someimplementations, the dynamic layout algorithms use an iterative processthat adjusts both the connectors and the visual marks until satisfactoryvisual effects are obtained.

FIGS. 10A-10H illustrate a sequence of data visualizations created by auser who is evaluating airline flight data between states in the UnitedStates. This scenario shows flows across geographic networks. For thisexample, the data in stored in a data source 236 with the following datafields 238 about flights:

-   -   Airline    -   FlightNum    -   DestinationState    -   DestinationAirport    -   OriginState    -   OriginAirport    -   #Passengers    -   Revenue

The user is an analyst for a major airline. The airline is consideringadding additional destinations and wants to understand the revenue andpassenger flow to all possible destinations. The user begins by creatingthe bar chart 1000 in FIG. 10A. The bar chart 1000 shows the totalnumber of passengers flying into each state. In the user interface 500,the DestinationState field is placed on the rows shelf 532 and theformula for the number of passengers is SUM([#Passengers]).

The airline already has routes to CA and TX, but IL and FL both seemlike interesting opportunities for expansion. The user is interested inseeing how these new destinations interact with their current routes sothe user switches the view type to a map view as illustrated in FIG.10B. In the map view, each DestinationState is correlated to a positionon the map, and visual marks are created for each state, such as themarks 1004 for California and Texas. The marks use a size encoding tovisually display the number of travelers to each state. The user doesthis by placing the formula SUM([#Passengers]) onto the size shelf 544.

This airline only operates in major population centers, so the userfilters out states with less than a specific number of passengerstraveling to them, as illustrated in FIG. 10C. Here, the user specifiesa filter using SUM([#Passengers]). In addition, the user also sets anexplicit filter to remove Georgia from the map because it is not adestination the airline can consider at this time. The application ofthe second filter is not illustrated in FIG. 10C.

The user then adds connectors to the map, including the connectors 1008illustrated in FIG. 10D. The connectors here are sometimes referred toas “To/From Edges” because they connect a source and a destination. Inmany cases there are pairs or connectors between two states,illustrating the travelers in both directions. The flight data definesthe relationship between source and destination, and that relationshipis placed on the connector shelf 536.

In some implementations, the user uses the relationship builder module232 to define the relationship with the source as [OriginState] and thedestination as DestinationState. In this data set, a row in the databasedirectly maps to a single edge in the graph. Each node (a visual mark)corresponds to one or more rows in the database. For example, Texas is asingle node in the graph, but there are flights into Texas from multipleother states.

This example raises a number of interesting issues. First, the sizeencoding for the marks applies only to the marks, and not to theconnectors. As seen in FIG. 10D, all of the connectors have the samewidth. Second, the filter that excludes GA only excludes Georgia as aDestinationState, while not excluding Georgia as an OriginState. As aresult, some implementations create a “ghost” node 1010 in thevisualization to represent the edge from Georgia to Florida.

The user is able to pick from a number of rendering styles for theedges, including straight edges between points, simple arcs, or greatcircle arcs. In this instance, the user has chosen the default straightedges.

In FIG. 10E, the user has removed the ghost node (e.g., by toggling aGhost Node display option in a display menu). In addition, the user haschanged the color of the edges using the color encoding shelf 556 forconnectors. Finally, the user has encoded the size of the connectors byplacing the formula SUM(#passengers) on the connector size encodingshelf 546. For connectors, the size is the width.

In FIG. 10F, the user has removed the connectors that represent toolittle traffic by applying an aggregate filter to the edges. Theaggregate filter for the connectors uses SUM(#Passengers)>Threshold,where the threshold is a specific value. This filter removes the edge1014, for example, which was previously displayed in FIG. 10E.

FIG. 10G illustrates filtering the nodes in a graph based on therelationship for the connectors. The user wants to limit the display tonodes that are connected to Florida in the current graph. In someimplementations, the user accomplishes this by right clicking on the FLnode and adding a filter to limit the display to nodes that areconnected to Florida by links that are still in the graph. This is anexample of filtering visual marks based on properties of the connectorsor a relationship. In this case, the New York node 1018 is now excluded.On the other hand, Colorado 1020 is still included because there is apath to Florida that goes through Texas 1022. Although not depicted inthis illustration, a node may remain in the display with sequences ofconnectors with more than two links.

FIG. 10H illustrates that that the user can split the data into smallmultiples based on the airline's two major competitors to see theirdifferent approaches to flying into Florida. In this case, the datapreviously displayed in FIG. 10G is split into two graphs based on theairlines, thus displaying a graph for Airline A 1026 and a separategraph for Airline B 1028. In some implementations, this splitting intopanes is accomplished by adding the airline data field to the columnsshelf 534. If the data included additional airlines, there would beadditional panes. Alternatively, if the user is interested in only thesetwo airlines, the user could add another filter.

After this work, the user can choose to go back to a bar chart, and seedata corresponding to the various filters and selections that have beenapplied. In some implementations, the user invokes a data visualizationrecommendation module 642 to identify alternative ways to view the data.

FIGS. 11A-11J illustrate a sequence of data visualizations created formarket basket analysis. The data for this scenario is from ahypothetical SuperStore data source 236 with the following data fields238:

-   -   Product    -   OrderID    -   Date    -   Price    -   Sales    -   Margin

A user wants to study which products frequently occur in the same orderand how this has varied over time and across promotions. The user startsby placing the [Product] field on the Level of Detail shelf. Rather thanoverlapping the products, some implementations stack items in a cell bydefault. Because no fields have been placed on the rows or columnsshelf, the product names just wrap around. Assuming the user hasselected text marks for the products, the initial graphic may appear asshown in FIG. 11A.

In FIG. 11B, the user has encoded Price as the size of each item (e.g.,using the size shelf 544). This produces a pseudo-tag cloud. Someimplementations include “tag cloud” as a view type, so selecting thatoption results in a true tag cloud. See, for example, graphic 740B inFIG. 7C. In some implementations, a tag cloud is created when the userselects a “packed” layout type and selects a specific view type.

In FIG. 11C, the user applies some filters to focus on just this year'ssales, so the data is filtered to YEAR([Date])=2011. In addition, theuser limits the products to those with price <=1000, which eliminates“Desktop.”

FIG. 11D illustrates switching from a Cartesian layout to a Circularlayout. In some implementations, the user makes the selection byclicking on an icon near the Rows and Columns shelves. In someimplementations, this is accomplished by using a toolbar icon or a menuitem. As illustrated in FIG. 11D, the Text marks have optional borders,allowing them to appear more like “marks”.

In FIG. 11E, the user switches from Text marks to Circle marks, selects[Product] as the label, and adds Price to the size shelf. In general,these operations do not alter the location of the products. In theillustration of FIG. 11E, the user has selected a different sort orderas well.

FIG. 11F illustrates adding connectors to the graph. In this case, theuser wants edges to show which products are purchased together. This canbe achieved by using an equivalence relationship 416. Two records fromthe data source 236 are “equivalent” if they have the same OrderID. Theuser can place the equivalence relationship on the connector shelf 536to display the edges. Alternatively, the user could place the OrderIDfield on the connector shelf 536 to achieve the same result (i.e.,resulting in using the OrderID field as an equivalence relationship).

In some implementations, an edge is added for each instance of therelationship by default. The user can use the connector level of detailshelf to specify the level of aggregation for the connectors. In thiscase, the user sets the level of detail for the connectors to be basedon the two products that are being connected. In addition, the user usessize encoding 546 for the connectors so that the size of each connectoris based on the aggregated number of relationship instances.

In some implementations, the aggregation of the tuples by product leadsto aggregation of the connectors, or at least this is the defaultbehavior. An edge is added for each pair of products that have a common[OrderID] and a single edge is defined by many tuples. Recall that the[Product] field was placed on the level of detail shelf, so the nodesare aggregated by product. Even with automatic aggregation of theconnectors, the user would still need to select the size encoding of theconnectors.

In FIG. 11F, the user has applied an alternative sort order to thenodes, displaying them in alphabetical order by product name. In thisexample, the nodes start alphabetically on the lower right and proceedclockwise around the circle. Because the items are arranged around thecircle alphabetically, there are random clusters.

In FIG. 11G, the user has applied a filter to remove edges that occurless than 10 times, which removes some of the “noise” in the graph. Forexample, the edges 1110 are filtered out. In some implementations, thisis accomplished by setting the formula SUM(Number of Records) >=10 as anaggregate connector filter.

In FIG. 11H, the user experiments with a different sort order, nowsorting the nodes by price to see if that exposes any patterns. Someimplementations enable sorting a dimension using a correspondingmeasure.

The data visualization in FIG. 11H reveals some structure: high-priceditems are rarely bought together except XBox and TVs, and thatcombination purchase does not occur often. The items in FIG. 11H arespaced equally around a circle, which is the default behavior. Someimplementations allow a user to specify the spacing using a formula orfield.

In FIG. 11I, the user has spaced the items by their price, which isaccomplished by adding SUM(Price) to the Rows shelf 532 (whichtranslates to the angle of the product around the circle in a Circularlayout). In some implementations, the labels “rows” and “columns” arechanged to reflect the actual usage in a Circular layout.

Here the user can see that the links between high-priced items and lowerpriced items are focused on a specific cluster of products which arecurrently offered on a promo of “50% off these items when included inorders >$200.” The user also sees the expected links between productslike a Camcorder and batteries.

The user is interested in knowing whether the promo that ran last year(“2 for the price of 1”) resulted in more high priced purchases at thesame time. As illustrated in FIG. 11J, the user filters the data toinclude last year (which is 2010 in this example) and adds YEAR(Date) tothe Columns shelf to split the data into two panes. As seen in FIG. 11J,some of the data is quite different between the two years. See, forexample, connectors 1120 in 2010 versus connectors 1120′ in 2011, andconnectors 1122 in 2010 versus connectors 1122′ in 2011.

FIGS. 11I and 11J also illustrate a difference in how ordinal andquantitative fields are interpreted when they are placed on the rowsshelf 532 or the columns shelf 534. In FIG. 11I, the quantitative fieldSUM(Price) was added to the Rows shelf, and it was interpreted tospecify the spacing. On the other hand, in FIG. 11J, the ordinal fieldYEAR(Date) was added to the columns shelf, and it resulted in splittingthe data visualization into panes, each with a separate graphic.

FIGS. 12A-12F illustrate some features of a data visualizationapplication 222 that may be applied to a social network. The source data236 includes one table that represents people, and a second table thatshows friendships between the people who play video games against eachother. The Person table includes these data fields:

-   -   Name    -   ImageURL    -   Gender    -   Age    -   Income    -   Company

The IsFriend table includes these data fields 238, where the Person1 andPerson2 fields match the names of people in the Person table.

-   -   Person1    -   Person2    -   FriendSinceDate    -   #GamesPlayedTogether

In some implementations, these two tables are used directly. The Persontable will be displayed as nodes in a graph, and the IsFriend tableestablishes a relationship, which can be used to create connectors forthe graph. In some implementations, the data visualization applicationallows a user to join these two tables, denormalizing them into a singleschema using a left outer join from the Person table to the IsFriend. Inthis case, the resulting single table may include the following datafields:

-   -   Name    -   ImageURL    -   Gender    -   Age    -   Income    -   Company    -   FriendName    -   FriendSinceDate    -   #GamesPlayedTogether

The user analyzing this data wants to create a community among theplayers of their video games. To do this, the user is interested in thecharacteristics of people who play together versus alone, and whatcharacteristics go together for people who play against others. The userstarts by creating a simple list of players by placing [Name] on thelevel of detail shelf, which results in a long wrapped and clipped listof player names as illustrated in FIG. 12A. By default, this is using aCartesian layout type. Here, the marks are Text marks.

In FIG. 12B, the user changes the layout type to “Network.” Becausethere are no edges displayed yet, the network layout places the Playersinto a packed layout. In the packed layout, the overall size isminimized, both vertically and horizontally.

In FIG. 12C, the user has used [Income] for size encoding (e.g., usingshelf 544). This repositions items in the packed layout, somewhat like atag cloud. except that in this case there are borders.

In FIG. 12D, the user has added connectors based on the relationprovided by the IsFriend table (or from the denormalized table). Thiscreates edges between people who have played video games against eachother. In this case, the edges are undirected. In some implementations,the source data is directed (e.g., the denormalized table describedabove), so the user selects undirected in the user interface. When edgesare added, the Network layout applies a force-directed layout algorithmto spread out the nodes, as seen in FIG. 12D.

The data visualization in FIG. 12D does not yet provide enough structureto see any patterns in the data. The user wants to introduce moresemantics into the placement of nodes within the graph. To investigatepotential correlation by age, the user adds [Age] to the columns shelf534. As shown in FIG. 12E, the Age field introduces an additional forceinto the force-directed layout, which pushes nodes corresponding toolder people to the right and nodes for younger people to the left. Onevisible correlation is that the older people also have higher incomes,but that correlation is not helpful here.

Note that this behavior of the columns shelf 534 for a network layout isquite different from the usage of the columns shelf in a Cartesianlayout. In a Cartesian layout, placing a quantitative field on thecolumns shelf creates a quantitative x-axis and encoding. In a networkor packed layout, the fields on the rows and columns shelves are used asinputs to the layout algorithm, creating additional forces that affectthe positioning of the nodes. In some implementations, for a network orpacked layout, the screen labels “rows” and “columns” are replaced withalternative labels that express the usage as creating vertical orhorizontal forces.

FIG. 12E shows that age is not a key factor in determining which peopleplay against each other. In particular, there are many edges thatstretch horizontally across the graph between older players to youngerplayers.

In FIG. 12F, the user investigates the role of gender by adding [Gender]to the rows shelf 532. The result is different from adding the Age fieldbecause of the different data type (Age is quantitative, whereas Genderis ordinal). As shown in FIG. 12E, the quantitative Age field can beused to apply numeric forces in the layout. But the Gender field has noquantitative interpretation. Instead, the ordinal Gender field splitsthe display into two panes vertically, with separate panes for “male”and “female.”

In the absence of edges, each pane is a self-contained graphic, and insome instances edges are not permitted to cross pane boundaries. Here,however, all of the men are in one pane and the women are in a secondpane, so the relationship requires edges that cross the pane boundary.

This view in FIG. 12F shows that Sally is the only female playing videogames against male players. There are no other edges that cross paneboundaries. In some implementations, the user can replace the Text markswith images of the people. In some instances, the images are stored aspart of the Person table, or the Person table includes links to storedimage files (e.g., URL's or file names). Each node is re-rendered toshow the corresponding profile picture.

FIGS. 13A-13D illustrate some interactive features that are availableafter a data visualization has been rendered. The data for theseexamples is for football games between college teams. FIG. 13A shows thestructure of the Team table 562 and the Game table 1302, and some of thedata in these tables. The Team table 562 was illustrated above in FIG.4E, including the team field 570, the division field 572, and thesubdivision field 574.

The game table 1302 provides information about individual games thathave been played. Each row in the game table 1302 include the date 1304the game was played, the home team 1306 (which corresponds to a team 570in the team table), the home score 1308, the away team 1310 (whichcorresponds to a different team 570 in the team table 562), and the awayscore 1312. The game table 1302 creates a relationship between rows inthe team table.

FIG. 13B illustrates the data visualization region 520 within userinterface 500, including both the shelves 524 from the interface 500 andthe graphic display region 530 displaying the data visualization. Inthis illustration, each of the teams is illustrated as a small circlewith its corresponding team name displayed nearby, and the connectorsrepresent games between the teams. The teams in each subdivision arelaid out with a Circular layout, and the divisions and subdivisions areorganized into rows and columns.

Some of the user selections are displayed on the shelves. In thisillustration, the user has specified GROUP({Division}) 1320 for therows, so each of the “rows” in the data visualization corresponds to agroup of divisions. The top row 1336 in the data visualization includesthe Atlantic Coast division.

For the columns, the user has specified WITHIN-GROUP({Division}) 1322,which indicates that the columns are based on the divisions within eachgroup of divisions. (See fields 784 and 790 in FIG. 7E.) For theconnection shelf, the user has specified the relationship 1324, which isbased on the game table 1302. In some implementations, the user canassign a more concise name for the relationship. In this case, theexpression “Scores” in the relation name 1324 indicates that the teamscores will be used for encoding the size of the connectors. Inparticular, this example uses tapered sizes, where the width of eachconnector at the endpoints is based on the scores of the teams, andgradually tapers between those two widths. In this example, the startingsize (width) of each connector is specified as the home score 1330, andthe ending size is specified as the away score 1334.

The text encoding for each node includes both the team name 1326 and therank 1328 of the team if it has a ranking. In addition, each mark (thecircles for each team) use size encoding based on the number of wins1332 (e.g., the number of wins within the team's conference).

After the data visualization has been rendered, some implementationsenable a user to interact with the data visualization to identifyspecific items of interest. For example, in this data visualization, theuser has highlighted three specific connectors, including the connectors1338. In some implementations, the user can highlight individual edgesby clicking anywhere on the edge. In some cases, where many edges aretightly packed, the user may use a zoom feature first so that it iseasier to identify the desired connector. The highlighted edges 1338illustrate that the game between LSU and Kentucky was very close, andboth teams had high scores. On the other hand, in the game betweenKentucky and Kent State, Kentucky won by a large margin.

Some implementations also provide context popups to provide additionalinformation about nodes or connectors. For example, in someimplementations, a user may get further information about a team or agame by right-clicking on a node or an edge. In some implementations,the additional information is displayed in bubbles based on hovering themouse cursor at a specific location for a designated length of time. Insome implementations, bubble popups are not displayed unless it is clearwhich item the user would want (e.g., when there 10 different edgesaround the location of the mouse cursor, there is no clear choice).

In some implementations, selecting a mark automatically highlights allof the connectors associated with the mark. In some implementations,selecting a connector highlights the marks associated with therelationship.

In some implementations, when the x and/or y position of a mark wasarbitrarily chosen by the data visualization generation module, dragginga mark moves the mark to a new position.

In some implementations, when an indented list is used for the labelscorresponding to a relationship (e.g., a tree), the “+” and “−” buttonsenable a user to expand or collapse portions of the hierarchy,effectively filtering the display. In some implementations, expand andcollapse buttons are provided on the data visualization itself forcertain graphs.

FIGS. 13C and 13D illustrate schematically how some implementationsprovide information detail that is based on the magnification level.(Implementations provide ordinary zoom-in and zoom out as well.) This isanother example of post-rendering interactive behavior. In FIG. 13C, thedata shown in FIG. 13B is displayed at a level of detail appropriate fora high level view. At this magnification, each division is shown as asingle mark 1350, such as a disc. The games played between teams withineach division are not depicted here, but some implementations providesome high level information, such as the number of connectors that are“inside” each mark, or other designated information. The high levelconnectors, such as connectors 1352 and 1354 indicate the games playedbetween teams from the different divisions. In some implementations, theconnectors 1352 and 1354 are encoded in various ways. For example, thesize may encode the number of games each connector represents, ortapering may be used to show the relative ranking of the divisions(e.g., in the aggregate).

FIG. 13D illustrates schematically the display after the user has zoomedin on a specific division. In FIG. 13D, games played against teams inother divisions are not displayed, but all of the games against otherteams in the same division are displayed. (In this illustration, not allof the games are depicted.) A zoom in view such as shown in FIG. 13Dtypically retains the encoding selected by the user, including theencodings of the nodes (e.g., size based on the number of wins and theteam ranking) and the encodings of the connectors (e.g., the tapering).As described above with respect to FIG. 13B, some implementationsprovide additional information about teams or games by clicking,hovering, or otherwise selecting a displayed element.

FIG. 13D also illustrates a layout in which the labels are placed toavoid overlap with the connectors. Here, the names of each team areplaced away from the connectors. For example, at the Boston College node1370, the connectors are all directed toward other team nodes in acircular layout, whereas the team name is located outside the circle.Some implementations apply this technique in other diagrams as well whenpossible.

Diagrams with connected relationships can get very large and complex.This can make it difficult to see anything specific, and can make thevisualization slow to draw. There are many ways to improve both thespeed and comprehension of the graphics that take advantage of theinteractivity of a computer. Some implementations provide one or more ofthese features, some of which are implemented as part of thepost-generation options 650:

-   -   filters on the marks and separate filters on the connectors;    -   color highlighting of particular connectors (like color legend        highlighting);    -   expand/collapse/prune portions of a tree (up or down) or        relationships from a node;    -   aggregation of relationships;    -   show a single node in the center surrounded by the nodes that it        is directly related to but no other nodes. Click on a node to        make it the center and reveal the new connections;    -   overview window with a zoom box;    -   during the generation of a data visualization, first show a        minimal network, then expand as work is fully generated;    -   animating changes in position and encoding to show how things        change;    -   zoom and pan, including interactive techniques such as fish-eye        displays;    -   user adjustment of the positions of points and the routing of        edges.

Many naturally occurring networks (such as human networks or computernetworks) tend to have hubs with a large number of connections ratherthan being pseudo-random. Some implementations identify these hubs bysetting a range filter on the number of connections per node, making iteasy to see nodes with large numbers of connections.

FIG. 14 lists some uses 1400 of relationships within datavisualizations. Relationships 240 provide many new ways to filter 1402data in a data visualization, even when the relationship is notdisplayed using connectors. In many cases, the relationship is a tree,or can be used to build a tree using a depth first search or breadthfirst search, as described above with respect to FIG. 7E. (Asillustrated in FIG. 7E, a relationship can be used to construct variousquantitative fields.)

Using a tree relationship, the nodes in a data visualization can befiltered in various ways. Some implementations enable a user to specifya node, and filter (1408) the display to include just that node and thenodes below it in the tree. Some implementations expand this in variousways, such as allowing a user to select multiple nodes and filtering tojust the subtrees below those nodes.

Some implementations allow a user to filter (1410) the nodes to aparticular depth in the tree. For example, the specified depth may be 3,in which case nodes with depth of 0 (the root node), 1, 2, or 3 arekept, and all lower nodes in the tree are filtered out. Note that theterm “tree” in this context is not necessarily fully connected, so theremay be multiple root nodes.

Some implementations allow filters that combine the two precedingconcepts, limiting the set of nodes to just those within a certain depthbelow a specified node. In some implementations, a relationship may becombined with an ordinary filter based on node properties to create amore complex filter. For example, consider a very large family treehierarchy, using data similar to that shown in FIG. 4E. Now, supposethat a set of people is identified by some set of criteria related tohealth, and a user wants to investigate the health of the relatedparents and children. In this case, the data is filtered to thoseindividuals satisfying the criteria as well as those who are related bybirth to a person satisfying the criteria. In general, implementationsallow filter expressions based on any number of properties of the nodesas well as on relationships and relationship properties.

Implementations allow users to filter connectors as well. For example, auser may filter (1414) connectors based on any connector properties. Inaddition, when a relationship is directed, a connector filter may useproperties of either the source or destination nodes. In addition, auser may filter (1412) connectors based on aggregate properties, such asthe number of connections between two nodes (i.e., the number ofindividual relationship instances between tuples).

Whereas nodes can exist by themselves, connectors need to connectsomething. Therefore, if the two nodes that would be the endpoints of aconnector are filtered out, the connector is implicitly filtered out aswell. When only one endpoint of a connector is filtered out, there issome ambiguity about whether the connector should be filtered out. Insome implementations, such a connector is always filtered out. In someimplementations, ghost nodes, such as node 1010 in FIG. 10D are allowed,so the connector remains in the display. In some implementations, thehandling of ghost nodes is configurable by the user, either forindividual data visualizations or for a user (e.g., as a user preference242).

A relationship can also be used to sort (1404) data within a datavisualization. As noted with respect to FIG. 7E, once a relationship isdefined, it creates a graph, and the graph can be traversed (1418). Insome instances, the traversal includes some arbitrary decisions, but theresult is a well-defined order of the entire set of nodes. Not only doesthis create an ordering of the whole graph, it creates an orderingwithin any subset, such as the nodes that are directly connected (1416)to a specified node. If the relationship is directed, and a selectednode is not a root, the direct connections may include both a parent andchildren.

In some instances, a relationship 240 can be used to aggregate (1406)tuples that create the visual marks or aggregate (1406) the visual marksthemselves. For example, in a tree with a single root, all tuples at thesame depth from the root may be aggregated. In another example, arelationship may partition the tuples into a plurality of groups(disconnected “islands”), so the relationship can be used to aggregatethe data for each of those groups. As a further example, nodes can becategorized based on the other nodes to which they are related (thenode's neighbors), and nodes with the same set of neighbors are groupedtogether. (Sometimes this aggregation is applied iteratively.) Ingeneral, nodes can be aggregated (1420) when the tuples have similarrelationships.

In some implementations, visual marks can be aggregated in aconsolidation process, where nodes placed at the same location aregrouped together. This may occur in a network layout when the nodes areplaced according to their relationships with other nodes.

In addition, connectors can be aggregated based on the nodes they areconnecting. In particular, a user may choose to aggregate (1422) allconnectors that connect the same pair of marks (or the tuples theyrepresent).

Some implementations use an alternative user interface to createconnectors for data visualizations. This alternative user interface isdescribed with respect to some specific examples illustrated in FIGS.15A, 15B, 16A, 16B, and 17A-17E. Although the user interface illustratedabove in FIG. 5A and the interface described below may be implemented asdistinct alternatives, one of skill in the art will recognize thatvarious features and aspects of the alternatives may be combined.

In some implementations, connectors are created as a connector layerusing a connector layout region of the user interface. In someimplementations, a user can select one of three classes of edges basedon the relationship that defines the edges.

A first class of edges is based on groups. In this scenario, the sourcedata defines the nodes. All nodes sharing a value for a specified field(or set of fields) form a group and are all connected to each other.This is another way of referring to an equivalence relationship 416. Anexample is Market-Basket analysis, where a relationship is defined byitems having the same value for an Order ID, as illustrated above inFIGS. 11A-11J.

A second class of edges is based on paths. In this scenario, the sourcedata defines the nodes and all nodes sharing a value for a specifiedfield (or set of fields) form a group. Within a group, the nodes areordered and connections only exist between adjacent nodes in thatordering. For example, in web analytics the data represents page visits.All pages with the same session ID form a group and within that groupthey are ordered by their time stamps.

A third class of edges is based on using a directed relationship. Inthis scenario, the data defines the edges and each tuple contains a “to”field and “from” field. The flights and social networking examples inFIGS. 10A-10H and 12A-12F, where each row defines a flight or a gameplayed together, are examples. A user needs to create marks to representthe nodes and then specify the edges. In general the edges are directed(from one node to another node), but some implementations expand this tocover scenarios that are undirected. This can be complicated when thenodes are defined by multiple dimensions (e.g., Year, Player, andGender) and only some of those fields matter for matching edges to nodes(e.g., Year and Player). This scenario is also complicated because theset of vertices (the nodes) is derived from multiple fields (e.g.,OrigAirport and DestAirport in the example of FIGS. 10A-10H). Asillustrated in FIG. 10D, this is also the case where you can get ghostnodes.

FIGS. 15A and 15B illustrate creating a data visualization with “group”edges. FIG. 15A illustrates the source data 236, which is an order table1500. Each record in the order table 1500 specifies a single line item,and multiple line items may be included in the same order. Thissimplified table includes the Date 1502 of the transaction and an OrderID 1504 for the transaction, which uniquely identifies the order. TheLineItem field 1506 uniquely identifies specific items within a singleorder. Each transaction also includes the Product 1508, the Price 1510,and the Quantity 1512.

To define the vertices for a node-link diagram, a user selects Circlemarks with [Product] on the level of detail shelf and a circular layout.Because the [Product] is on the level of detail shelf, only one node iscreated for each product. To create edges, the user adds a connectorlayer with edges of the type “Group.” This type of edge supports thefollowing properties and encodings:

-   -   Group. The user can select which data fields define the        grouping. This can be one or more fields. This is an alternative        way of describing an equivalence relationship 416;    -   Color. The user can select a data field that will determine the        color of each edge; and    -   Size (width). The user can select a data field that will        determine the width of each edge.

To define the edges, the user simply puts [OrderID] on the Group shelf.The data visualization application 222 then generates edge tuples 1520,as illustrated in FIG. 15B. Each edge tuple includes Product1 1522,Product2 1524, and the Number of records 1526. The number of records1526 is the number of instances where each product pair occurs as lineitems in the same order. The edges connect the appropriate nodes.

The nature of the group edges limits the set of choices for encodings ofthose edges. As noted above, implementations typically support encodingsfor color and size of the edges, but there are a limited number ofchoices for the fields that may be used. One available field is Numberof Records 1526, which is commonly used for Size encoding. In addition,the fields used in the grouping may be used in the encoding, becausethere is no ambiguity about their field values. Specifically, becauseall of the nodes within a group have the same values for the groupingfields, edges between two nodes can refer to the value of those fields.

On the other hand, other field values are inherently ambiguous becausean edge connects together two nodes with potentially different fieldvalues. In some implementations, when a user selects a quantitativefield (e.g., price) for edge encoding, and the quantitative field is notone of the grouping fields, the data visualization application computesthe average for the quantitative field (either the average for theentire group or the average for the pair or nodes that each edgeconnects). In the specific example illustrated in FIGS. 15A and 15B,this average works well because the prices for the products do notchange during the time interval of interest.

FIGS. 16A and 16B illustrate creating a data visualization with pathedges. FIG. 16A provides a table 1600 that is the data source 236 forthis example. Each row in the table 1600 represents a visit to a webpage, and multiple web page visits may be part of the same session. TheSessionID 1604 is a unique identifier for a browsing session. Withineach session, each web page visit has a unique timestamp 1602, which mayrepresent an actual time, or may be a sequential number within thesession. (The only requirement is that the values are monotonicallyincreasing within each session.) The URL 1606 specifies the web pagevisited. For this example, the URL names have been simplified. Thistable 1600 also includes a field 1608 that specifies the TypeOfPage andthe field SizeOfPage 1610, which indicates the size of the page (e.g.,in bytes or kilobytes).

To define the vertices of a node-link diagram, the user selects avisualization using the Text mark, puts [URL] on the Text shelf, andselects a circular layout. To create edges, the user adds a connectorlayer of the type “Path Edge.” This type of edge supports the followingencodings:

-   -   Group. The user can select which data fields define the        grouping. This can be one or more fields;    -   Ordering. The user can select which data fields define the        ordering. This can be one or more fields;    -   Color. The user can select a data field that will determine the        color of each edge; and    -   Size (width). The user can select a data field that will        determine the width of each edge.

To define the edges, the user places the [SessionID] field 1604 on theGroup shelf and [Timestamp] 1602 on the Ordering shelf. The datavisualization generation module 228 generates edge tuples 1620 asillustrated in FIG. 16B. The SessionID1 field 1622 corresponds to theshared SessionID 1604 of the tuples grouped together. The Timestamp1field 1624 and URL1 field 1626 correspond to the Timestamp 1602 and URL1606 of a first tuple. The Timestamp2 field 1628 and URL2 field 1630correspond to the Timestamp 1602 and URL 1606 of a second tuple. Foreach record in the edge tuples table 1620, the Timestamp1 field 1624 isless than the Timestamp2 field 1628. In addition, the value of theTimestamp2 field 1628 follows the value of the Timestamp1 field 1624according to the timestamps 1602 in the base data. For example, if a webuser went from web page A to page B then to page C, there will berecords in the edge table 1620 to show the transitions from A to B andfrom B to C, but there is no record from A to C. Because edges are basedon the ordering, the Number of Records 1632 in the edge tuples 1620 isalways 1. A user cannot transition from one web page to two differentweb pages simultaneously.

Using the edge tuples 1620, the data visualization generation module 228displays edges that correspond to the edge tuples. Unlike the exampleabove with respect to FIGS. 15A and 15B, the edges created here aredirected, having a specific source and destination.

Path edges provide greater flexibility for encoding because the orderingallows a user to specify whether to get data from the source node tupleor the destination node tuple. For example, if the user wanted to encodecolor based on the TypeOfPage 1608 or the SizeOfPage 1610, the usercould drop the field onto the appropriate shelf, and select which end ofeach edge to use. In some implementations, the user interface promptsthe user to make the selection, which may be displayed on a shelf toindicate the selection. For example, “TypeOfPage (source)” is used insome implementations to indicate that an encoding is based on theTypeOfPage of the source node. In some implementations, the suffixes“(from)” or “(to)” are used to indicate whether to use the source nodeor the destination node, such as “SizeOfPage (to).”

This additional flexibility means that Path edges can be encoded usingany fields defining the groups (just like group edges), the [Number ofRecords] (just like group edges), and any other fields in the nodetuples as long as the user specifies which node to use.

Because of the similarities between group edges and path edges, someuser interfaces combine these two concepts, and include an optionalordering encoding.

FIGS. 17A-17E illustrate creating a data visualization with to/fromedges. FIG. 17A provides a table 1700 of source data 236 used in thisexample. The table represents people playing video games against onlineopponents. The table 1700 includes a Person field 1702, which is thename of one of the players. The table 1700 also includes the Gender 1704corresponding to the Person 1702. The Table 1700 specifies theDatePlayed 1706, the name of the Opponent 1708, and the duration of thevideo game (DurationMinutes 1710).

In this scenario, the data represents edges, not vertices. In someinstances, this table 1700 has been constructed by the user (eitherusing the data visualization application 222 or another application).For example, there may be a normalized table that represents people (thevertices) and a separate normalized table that represents games (theedges). The user may have denormalized these tables into a single tableusing a left (inner) join.

Although the data in table 1700 specifies a direction (a Person and anOpponent), the edges are treated as undirected because a video game ismutual. In some implementations, the user specifies whether an edge isdirected (and the direction) by specifying the edge type. This may bespecified either in a user interface region for defining marks or usingan option on the edge (e.g., right clicking on an edge and specifyingthe selection in a popup window).

In this scenario, it is easy to identify the edges because theycorrespond to rows in the table 1700. However, identifying the nodetuples is more difficult because of the way the data is structured. Forexample, Sally is identified in the table 1700 only as an Opponent 1708,and has no gender specified in the table.

A user wants to construct a node-link diagram of the players who haveplayed against each other. The user places [Person] on the level ofdetail shelf, selects a Circular layout, and then selects to/from edgesfor the connectors layer. The user specifies that the edges areundirected and indicates how to identify the other endpoint for eachedge. In some implementations, the interface for the connector layerprovides an endpoint shelf when to/from edges are selected. Using thetable 1700, the user places [Opponent] 1708 on the endpoint shelf.

To compute the set of nodes to display, the data visualizationgeneration module 228 takes the union of the [Person] field 1702 and[Opponent] field 1708, as illustrated in FIG. 17B, creating a Persontable 1720. The union is required in order to get all of the players(including Sally).

The union, however, only includes the player names. Consider a user whowants to use [Gender] 1704 as the color encoding of the nodes. Thesource data table 1700 does not specify the gender for Sally, so theunion generates a NULL value for her gender. This is illustrated by themodified table 1720′ in FIG. 17C, which has the NULL value 1722 forSally's gender. Because of this, the color encoding would encode asexpected for the other players, but have no color encoding for Sally. Auser viewing the source data would generally be able to understand themissing data. In some implementations, the user interface providesinformation to explain the missing encoding.

In some cases, the user may be able to modify the source data in orderto get the desired encoding, as illustrated in FIGS. 17D and 17E. Insome instances, the user could add data to the source table 1700 tocreate an extended source table 1740, as illustrated in FIG. 17D. If thetable 1700 was created by a left inner join between a person table and agame table, the user could create table 1740 by using a left outer joininstead.

As an alternative, the user could construct an original source table1760 that includes OpponentGender 1762, as illustrated in FIG. 17E.However, this would create additional complexity for the user, becausethe user would need to specify that [Gender] 1704 and [OpponentGender]1762 are related fields, and this creates the opportunity to haveinconsistent data (e.g., does the Gender of a person when identified asa Person match the OpponentGender of the same person when identified asan Opponent).

An additional complexity arises when two or more fields define therelationship. Consider a data visualization that a user has created withYEAR(DatePlayed) on the Columns shelf, [Gender] for Color encoding, andPerson as the level of detail. Note that placing YEAR(DatePlayed) on theColumns shelf splits the data visualization into multiple panes based onyear (spread out horizontally into different columns). In this example,the source data 1700 has data for 2011 and 2012, so there will be twopanes. The user has created edges using the Person field 1702 andOpponent field 1708.

Consider the relationship defined by (Tim, Male, 2011, Sally). The Timvertex appears in both panes in the visualization so the application 222must determine which node to connect to the Sally vertex. Based onlooking at the source data 1700, it should be the Tim node in 2011. Butthis means that the relationship is not defined by [Person] 1702 and[Opponent] 1708 alone, but also by [Year]. On the other hand,[Gender]1704 is not part of the relationship when determining whichnodes to connect. Some implementations recognize the difference because[Gender] 1704 is a property of the nodes, whereas [Year] is a propertyof the relationship.

In some implementations, the user interface for a To/From edge displaysall of the dimensions in play as “drop spots.” The user places fields ontop of the drop spots to establish mappings. In this case the shelf hasa user interface showing Person, YEAR(DatePlayed), and Gender. The userplaces [Opponent] 1708 in a drop spot corresponding to the Person field1702 to indicate it is part of the relationship. At this point, therelationship is determined by just these two fields, and does notinvolve any other fields. In particular, all instances of Tim would beconnected to all instances of Sally. If the user wants YEAR to beincluded, the user can add YEAR(DatePlayed) to a drop spot to beincluded in the relationship. In this way, edges are defined byprojection onto these fields. Any field can be encoded as color or sizeproperties of an edge because these fields are added to the projectionof the edge table.

In some implementations, data visualizations with To/From edges arebased on different data sources 236 for nodes and edges. That is, onedata source to create all of the nodes and a distinct data source todefine the edges. In some implementations, a single visual specification234 is used to specify both the node data and link data. In someinstances, the data sources for the nodes or edges are blended from twoor more data sources.

As described with respect to FIGS. 15A, 15B, 16A, 16B, and 17A-17E, someimplementations add edges using layers. In these implementations, edgemarks are distinct marks on layers above the marks defining the nodes ina graph. Edges can be drawn between any two marks, even if those marksare not within the same pane.

Some implementations support one or more of these features:

-   -   filters that only apply to certain sets of marks (e.g., a layer        or pane). This will apply to all views with multiple layers        (e.g., dual axes, etc.);    -   tooltip definitions are specified “per pane” so that different        tooltips can be defined for edges versus nodes. In some        implementations the tooltips are editable by end users; and    -   multiple fields may be selected by a user for “Label” or “Text”        encoding and the user may edit the label. For example, a user        may place both [State] and [Airport] on the Text shelf and        format to get a data label like “WA: SEA.”

It is common in areas such as social network analysis to visualize andfilter data based on properties of a graph. For example, nodes are oftensized by their degree or the size of their subgraph. Other moresophisticated metrics such as betweenness centrality, closenesscentrality, and clustering coefficients are common. Some implementationsallow users to define these type of metrics and enable use of thesemetrics in calculations or in filters.

Some implementations provide a set of Quick Graph Calculations. When auser has constructed a node-link diagram, the user can select from amenu of these predefined formulas and expressions, or use one of thesepredefined formulas or expressions to build a more complex expression.In some implementations, the Quick Graph Calculations are available witha toolbar icon or from a Data Window, making it easy to drag anexpression onto the canvas as an encoding.

For node-link diagrams and dual axes views, the marks are at differentlevels of detail. This can be problematic, because weights assigned toedges will generally be smaller than weights assigned to nodes. Filterscan be scoped to a single layer or pane. In some implementations,setting up filters uses a user interface region for the marks (e.g., the“Marks Card”). The user may designate a filter by placing it on a dropspot for filtering.

Node-link graphs often have many labels, so label placement isimportant. A high priority for the data visualization generation module228 is avoiding label-label occlusion, making labels readable, andplacing as many labels as possible on the display. It is not asimportant to avoid label-mark or label-edge collisions. In someimplementations, label placement wraps text in order to better fit thenatural placement of the label.

FIG. 18 illustrates how data from two non-homogeneous data sources maybe blended together. The blended data may be subsequently used as thesource data for creating visual marks, connectors, or both.

In FIG. 18, there are two source tables 1802 and 1804, which have somecorresponding data fields 1806. The two source tables need not be storedat the same location or in the same format. For example, one of thetables may be in a SQL database on a server and the other table may bestored as a spreadsheet or CSV file on a local computing device. Thecorresponding data fields 1806 between the two tables 1802 and 1804 mayhave the same field names, but that is not required. Someimplementations provide a mapping tool so that a user may identify howthe fields are matched. In some instances, the data types of the sharedfields 1806 are identical between the two tables, but that is notrequired. Here, they must be compatible data types. For example, if oneof the tables uses a 25 character fixed length string to store a nameand the other table uses variable length strings to specify names, thetwo fields are compatible (e.g., using a variable length string whosemaximum length is at least 25 characters). Similarly, many differentnumeric types are compatible by converting to the data type with higherprecision. The blended table that combines tables 1802 and 1804 thusincludes fields 1820, . . . , 1822, with appropriate field names anddata types.

In addition to the shared fields 1806, the first table 1802 may includesome fields 1808 (e.g., fields 1816, . . . , 1818) that are not present(1812) in the second table 1804. These fields are included in theblended table, using the field names and data types as specified in thefirst table 1802. Conversely, the second table 1804 may include somedata fields 1814 (e.g., fields 1824, . . . , 1826) that are not present(1810) in the first table 1802. These fields are included in the blendedtable, using the field names and data types as specified in the secondtable 1804. For the data that is “missing,” null or blank values areused.

In addition to the shared fields 1806, the fields 1808 that are only inthe first table 1802, and the fields 1814 that are only in the secondtable 1804, a table ID field 1828 is added, so that the source of eachrow in the blended table is identified. For example, when blending twotables, the Table ID values may be “1” and “2.” In some implementations,the Table ID values are in a user friendly format, such as valuesspecified by the user.

Once two or more tables have been blended, the data may be used almostlike any other data source, keeping in mind that some data is missing.If a user is only interested in the fields 1806 that are shared betweenthe two tables, then there is no problem at all. The resulting table hasdata just like any other table. In some implementations, relationshipsare limited to using the shared data fields 1806. Other implementationsallow any relationship using any of the fields in the blended table. Ifa relationship uses a field that comes from only one of the tables, theconnectors will involve only nodes from that one table. Note that theuser can use two or more relationships in a single data visualization(see, e.g., FIG. 8K), so one relationship may connect nodes from asingle table while a second relationship connects nodes that may involveboth tables.

A blended table as illustrated in FIG. 18 may also be used to constructa graph with non-homogeneous nodes. For example, the first table 1802may represent people and the second table 1804 may represent businessentities. They may share certain characteristics, such as having a name,an address, and a tax ID.

FIGS. 19A and 19B provide a flowchart for a process 1900 of generating(1902) a graphical representation of one or more data sources. Theprocess is performed (1904) by a computer having one or more processorsand memory. The process begins by generating (1906) and displaying(1906) a graphical user interface on a computer display, such as theuser interface 500 illustrated in FIG. 5A.

The graphical user interface 500 includes (1908) a schema informationregion 510 and a data visualization region 520. The schema informationregion includes (1910) multiple field names, where each field name isassociated with a data field from the data source. In the example userinterface 500 in FIG. 5A, the field names are displayed in a dimensionssection 502 and a measures section 504 of the schema information region510. As illustrated in FIG. 5A, the field names may include a computedfield (such as OVERALL({ID=father_id}). This example illustrates that insome instances, a first field name (of the multiple field names)identifies (1912) a computed field whose value for each tuple iscomputed based on an associated data field from the data source and afirst relationship. In the specific example of OVERALL({ID=father_id}),it is associated with the data fields ID and father_id, as well as therelationship {ID=father_id}. As explained above with respect to FIG. 7E,the computed values for OVERALL({ID=father_id}) are based on (1914) atraversal of a graph corresponding to the tuples and the firstrelationship.

In addition to the field names, the schema information region 510includes (1916) one or more relationship names, where each relationshipname is associated with a relation between rows of the data source. Inthe example in FIG. 5A, the relationship names are {ID=father_id} and{ID=mother_id}, which correspond to father-child and mother-childrelations between rows in the data source (e.g., rows in the family treedata 438, illustrated in FIG. 4C).

As illustrated in FIG. 5A, the data visualization region 520 includes(1918) a plurality of shelves, including a row shelf 532, a column shelf534, and a connector shelf 536. Although these shelves are depicted inFIG. 5A at specific locations and in a specific arrangement, one ofskill in the art will recognize that many other configurations arepossible. To accommodate some smaller display screens, someimplementations display only portions of the user interface, with popupsor other windows provided as needed.

In addition to the row shelf 532, column shelf 534, and connector shelf536, the data visualization region 520 typically includes other shelvesto specify properties of the visual marks (e.g., text shelf 542, colorshelf 552, size shelf 544, and shape shelf 554) and properties of theconnectors (e.g., size shelf 546 and color shelf 556).

To define a data visualization, a user associates data field and/orrelationships with various shelves that control various aspects of adata visualization. Commonly, this is performed by dragging and droppingthe data fields or relationships onto the shelves. One of skill in theart will recognize that a graphical user interface can provide variousways to associate a field or relationship with a shelf.

The process 1900 detects (1920) a user selection of one or more of thefield names and a user request to associate each user selected fieldname with a respective shelf in the data visualization region. Typicallya user associates a field name with a shelf one at a time (e.g., usingdrag and drop). A user can also disassociate a field name from a shelf(e.g., by selecting the field name on a shelf and pressing the deletebutton on the keyboard). In some instances, one of the field names isassociated with the row shelf of the column shelf. In particular, afirst computed data field may be associated with the row shelf or columnshelf (see., e.g., row shelf 532 and column shelf 534 in FIG. 5A).

The process 1900 also detects (1924) a user selection of one or more ofthe relationship names and a user request to associate eachuser-selected relationship name with a respective shelf in the datavisualization region. Like data fields, relationship names are typicallyplaced one at a time, and commonly associated with shelves using dragand drop. As explained above with respect to FIG. 5A, both data fields238 and relationship 240 can generally be associated with any of theshelves (with only a few limitations). In some instances, a firstrelationship name is associated with (1926) the column shelf or rowshelf by the user (see, e.g., FIG. 8H).

The process generates (1928) a visual graphic in accordance with therespective associations between the user-selected field names andcorresponding shelves and in accordance with the respective associationsbetween the user-selected relationship names and corresponding shelves,and displays the visual graphic in the data visualization region. Thatis, the user selections determine what data is displayed and how it isdisplayed. In some instances, the visual graphic includes (1930) visualmarks corresponding to retrieved tuples from the data source. The visualmarks can take many different forms, including dots, bars, text, boxes,shapes, and so on based on user selection. In some instances, verticaland horizontal placement of the visual marks is based on (1932) itemsassociated with the row shelf or column shelf by the user (the row shelfdetermining the vertical placement and the column shelf determininghorizontal placement). In some instances, each of the items is a fieldname or relationship name.

In some implementations, the visual graphic includes (1936) edges thatconnect the visual marks. This has been illustrated in many of thefigures above, including the data visualizations in FIGS. 8A-8E and8F-8K. In some instances, the edges correspond to (1938) a relationshipname associated with the connector shelf by the user. For example, inFIG. 5A, the relationship name {ID=father_id} on the connector shelf 536creates the connectors 560.

In some implementations, the edges correspond to a field name associatedwith the connector shelf by the user. In this case, the field name istreated as an equivalence relationship 416, connecting all marks whosecorresponding tuples have the same value for that field. In particular,each edge connects (1942) two visual marks whose corresponding tuplesshare a same field value for the field name.

In some implementations, the horizontal or vertical placement of visualmarks is determined (1944) by a user-selected function of the tuplesbased on a traversal of a graph corresponding to the tuples and thefirst relationship. This is illustrated in FIG. 5A, where the horizontalplacement (corresponding to the column shelf 534) uses the expressionDepth({ID=father_id}). The Depth function computes the depth for eachtuple in a tree formed by the hierarchy data. The depth is based on atraversal of a graph corresponding to the relationship.

In some implementations, the data visualization region includes one ormore connector property shelves. In some instances, the process detectsa user selection of a relationship name or a field name and a userrequest to associate the user-selected relationship name or field namewith a first connector property shelf. In this case, generating thevisual graphic includes visually formatting the connectors in accordancewith the user selected relationship name or field name for the firstconnector property shelf. For example, using the data from FIG. 4D, thecolor or size of the connectors may be encoded based on the size of eachshipment, the item shipped, or the carrier.

FIGS. 20A and 20B provide a flowchart for a process 2000 forconstructing (2002) data visualizations using data from one or more datasources. Some aspects of this process are illustrated above with respectto FIGS. 6A and 6B. For example, the user interactions 622 implicitlycreate a visual specification 234, which is used by the datavisualization generation module 228 to generate and display (646) thedesired data visualization. The process is performed (2004) by acomputer having one or more processors and memory. The process begins byreceiving (2006) a visual specification 234, where the visualspecification includes a plurality of properties and correspondinguser-selected property values, which define a data visualization layout.In some implementations, the visual specification is received from auser interface, such as the one illustrated in FIG. 5A. A first propertyvalue of the user selected property values identifies (2008) one or moresource databases for the data visualization to be generated anddisplayed. In some instances, two or more source databases are combined,as illustrated above with respect to FIG. 18. In some instances, onedatabase is specified for retrieval of node data and a distinct database(or a distinct table in the same database) is specified for retrieval oflink data, which will be used to generate edges or connectors betweenthe nodes.

Using the visual specification, the process determines (2010) one ormore node queries corresponding to one or more data fields in the sourcedatabases. In some instances, the retrieval process constructs datafields based on raw data in the data source. For example, if anOrderDate field exists in the data source, the visual specification mayspecify YEAR(OrderDate) as a data field to retrieve. In some instances,the construction of the data field YEAR(OrderDate) is performed by thedata source as part of the retrieval (e.g., using an SQL query to arelational database). In other instances, the data field OrderDate isretrieved from the data source, and the new data field YEAR(OrderDate)is computed locally by the data visualization application when needed.

The process 2000 also determines (2012) one or more link queries fromthe visual specification. The link queries correspond to (2012) a firstrelationship between rows of the source databases. In some instances,the first relationship is user-selected from a predefined set ofrelationships. In some instances, the first relationship is inferredbased on user selection of a data field (e.g., automatically building anequivalence relationship). In some instances, the first relationship isconstructed and saved by a user, in which case it behaves essentiallythe same as a predefined relationship. In some instances, a userconstructs a relationship that is stored only with an individual visualspecification. Even though it is a “single-use” relationship, therelationship in known in the visual specification, and thus behaves likea predefined relationship.

In some instances, the first relationship is user-selected (2014) from apredefined set of relationships, and the one or more of the link queriesare constructed (2014) from the first relationship. In some instances,the first relationship is an equivalence relationship 416. The firstrelationship corresponds (2016) to a specific data field in the sourcedatabase. Two rows of the source database are related (2016) by therelationship when the two rows have the same field value for thespecific data field. In some instances, the first relationship is afirst-order relationship 410. In this case, the first relationshipcorresponds (2018) to a first field f and a second field g, both ofwhich are data fields in the source database. A first row of the sourcedatabase is related (2018) to a second row of the source database when afield value for field f in the first row equals a field value for thefield g in the second row. Equivalence relationships 416 and first orderrelationships 410 are two types of relationships that may be identifiedor defined for a given data source, but there are many other types ofrelationships as well, as described above. For example, FIG. 4Bdescribes many types of relationships, and FIG. 4E illustrates aspecific example of a category tree relationship.

In some instances, the one or more link queries are constructed (2020)from a user selected field in the source database. The link tuples arepairs of rows in the database that have a common value for the userselected field. In this case, the selected field has created an implicitequivalence relationship.

Using the node queries, the process retrieves (2022) a plurality of nodetuples from the database. Each node tuple satisfies (2022) at least oneof the node queries. Similarly, using the link queries, the processretrieves (2024) a plurality of link tuples from the database. Each linktuple satisfies (2024) at least one of the link queries. The node tuplescorrespond to visual marks and the link tuples correspond to connectorsbetween the visible marks.

The process 2000 generates (2026) and displays (2026) visual marks inthe data visualization corresponding to the retrieved node tuples. Asillustrated in FIGS. 7B-7D, for example, the visible marks may be drawnin many different forms. Typically, the x and y positions of the visiblemarks are specified in the visual specification (e.g., originating froma user's selection on the rows shelf 532 and the columns shelf 534). Insome instances, the horizontal placement of visual marks is determined(2028) by a user-selected function of the node tuples based on atraversal of a graph corresponding to the node tuples and a secondrelationship specified by a property in the visual specification. Thiswas illustrated above in FIG. 5A, where the horizontal location of themarks is specified by the Depth( ) function and the relationship{ID=father_id}. In the example of FIG. 5A, the depth within the familytree hierarchy determines the horizontal positions of the marks (theboxes for each person) in the data visualization 540. Determiningplacement based on a function and a relationship can be applied tovertical placement as well. In some instances, the first relationship isthe same as the second relationship.

In addition to the vertical marks, the process 2000 generates (2030) anddisplays (2030) edge marks (connectors) in the data visualizationcorresponding to the retrieved link tuples. Each edge mark visuallyconnects (2030) a pair of visual marks corresponding to the node tuples.This is illustrated, for example, by FIGS. 11E and 11F. The datavisualization in FIG. 11E is based on a visual specification thatspecifies only the nodes, and in FIG. 11F connectors have been added.

In some instances, the data visualization is subdivided (2032) into aplurality of panes based on the visual specification. Each pane includes(2032) a plurality of visual marks and a plurality of edge marks. Thisis illustrated above, for example, in FIGS. 11I and 11J. In FIG. 11Ithere is a single pane, with all of the nodes and connectors in that onepane. In FIG. 11J, however, the visual specification has split the datainto two panes based on the year (a pane for 2010 and a separate panefor 2011). In the example of FIG. 11J, each edge mark connects (2034) apair of visual marks within a single pane. In other instances, at leastone edge mark connects (2036) a pair of visual marks that are indistinct panes. This is illustrated above in FIG. 12F, where there areseparate panes for male and female players, and some of the connectorscross a pane boundary to show a video game played between a male playerand a female player.

FIGS. 21A-21C provide a flowchart for a process 2100 for filtering(2102) data in data visualizations. Some aspects of this process areillustrated above with respect to FIGS. 6A, 6B, 8I, 10B-10G, 11B-11G,and 14. The process 2100 is performed (2104) by a computing device 102having one or more processors and memory.

The process 2100 retrieves (2106) a set of tuples from a databaseaccording to user selection, where each tuple includes the same set offields. In some implementations, all of the tuples have (2108) the samestructure, including the number of fields in each tuple, the order offields in each tuple, the data types of the fields, and the field names.In some implementations, some of the tuples include additional fields,or have the fields arranged in a different order. In someimplementations, the fields in the tuples do not have field names, andare identified based on their order within the tuples (e.g., the firstelement of each tuple corresponds to the same data field from the datasource). In some implementations, the data types of corresponding fieldsin the tuples are identical (e.g., the first field in every tuple is adouble precision floating point number). In some implementations, thedata types of corresponding fields in the tuples are not necessarilyidentical, but are instead required to be compatible (e.g., the secondelement in every tuple is either a 32-bit integer or a 64-bit integer).

The process 2100 identifies (2110) a relation between tuples. A relationis a well-defined rule that specifies whether a pair of tuples isrelated. For an ordered relation, the order of the two tuples in a paircan make a difference. A relation can also be considered (2110) anon-empty set of ordered pairs of tuples from the set of tuples. The setof ordered pairs identifies the tuples that are related. FIG. 4Bidentifies some types of relations and how relations can be classified.As noted previously, the terms “relation” and “relationship” may be usedinterchangeably.

In some instances, the relation is (2112) an equivalence relation 416.In this case, the relation corresponds to (2112) a field fin the set offields. The relation consists of ordered pairs of distinct tuples (t₁,t₂) for which t₁ and t₂ have a same field value for the field f.

In some instances, the relation is a delta-tolerance relation 418. Inthis case, the relation corresponds to (2114) a field fin the set offields and a positive number δ. The relation consists of ordered pairsof distinct tuples (t₁, t₂) for which f field values f₁ and f₂corresponding to tuples t₁ and t₂ satisfy |f₁−f₂|<δ.

In some instances, the relation is a first-order relation 410. In thiscase, the relation corresponds to (2116) a first field f and a secondfield g that are both in the set of fields. The relation consists ofordered pairs of distinct tuples (t₁, t₂) for which the f field valuefor t₁ equals the g field value for t₂.

The process receives (2118) selection of one or more filter conditionsfor the tuples, where at least one of the filter conditions uses therelation. In some instances, the filter conditions use one or more“base” tuples, and filter the remaining tuples to those that are withina certain “distance” of one of the base tuples based on the relation.The distance is the number of “edges” or links that must be traversed ina hypothetical node-link diagram where the tuples correspond to nodesand each edge corresponds to a relation between a pair of tuples. Notethat the data visualization to be displayed is not necessarily anode-link diagram.

In some instances, the one or more filter conditions include (2120) afilter condition that limits the set of tuples to those tuples that areconnected to a selected base tuple. In some instances, a respectivetuple is connected to the selected base tuple when (2122) there is anon-negative integer n and a sequence of tuples t₀, t₁, . . . , t_(n)with t₀=the respective tuple, t_(n)=the selected base tuple, and(t_(i-1), t_(i)) is in the relation for i=1, 2, . . . , n. In thisscenario, there is a path from the respective tuple to the base tuple.In this case, the base tuple itself is considered connected to the basetuple using a path of length 0 (n=0).

In some instances, a respective tuple is connected to the selected basetuple when (2126) there is a non-negative integer n and a sequence oftuples t₀, t₁, . . . , t_(n) with t₀=the selected base tuple, t_(n)=therespective tuple, and (t_(i-1), t_(i)) is in the relation for i=1, 2, .. . , n. As above, the base tuple is connected to itself using a path oflength 0. In this scenario, there is a path from the base tuple to therespective tuple.

In some instances, there are multiple base tuples, and the one or morefilter conditions include (2128) a filter condition that limits the setof tuples to those tuples that are connected to one or more base tuples.The tuples in the set of base tuples are (2128) those that satisfy auser-defined rule involving fields from the tuples. For example, FIGS.11A-11J provide a data visualization for items purchased at a store. Inthis scenario, the base tuples could be designated as those items with aprice greater than $500. The tuples for the data visualization couldthen be limited to those that are connected to one of those base tuplesby a single link (plus the base tuples themselves).

In some instances, a respective tuple is connected to a base tuple when(2130) there is a non-negative integer n and a sequence of tuples t₀,t₁, . . . , t_(n) with t₀=the respective tuple, t_(n)=the base tuple,and (t_(i-1), t_(i)) is in the relation for i=1, 2, . . . , n. In otherinstances, a respective tuple is connected to a base tuple when there isa path in the opposite direction (from a base tuple to the respectivetuple).

In each of the cases identified above, the number n may be limited(2124) by a fixed positive integer N. For example, the set of tuples maybe limited to those that are within 2 links of a base tuple.

These examples may be combined in various ways. In particular, the setof base tuples may be limited to a single tuple, or there may be aplurality of base tuples; the paths of connectedness may go from a basetuple to a respective tuple, or may go from a respective tuple to a basetuple; and the number of links between base tuples and respective tuplesmay be limited to a fixed positive integer N (i.e., number of links ≤N),or the path lengths may be unlimited. In general, each base tuple isconsidered connected to itself.

The process 2100 receives (2132) a selection of an aggregation level,which includes one or more fields from the set of tuples. The fieldsincluded in the aggregation level effectively act like the fields in aGROUP BY clause of an SQL query.

Some implementations support receiving (2134) selection of an aggregatefilter condition that is applied to the aggregated tuples and based onthe relation. Whereas the filter conditions described above for theprocess 2100 apply to individual rows or records from the data source,an aggregate filter condition applies after the data has beenaggregated. An aggregate filter condition is similar to a HAVING clausein an SQL query. Here, however, the aggregate filter condition is basedon the relation. In some instances, the aggregate filter conditionlimits (2136) the set of aggregated tuples to those with at least aminimum number of connections to other aggregated tuples. This isillustrated above with respect to FIGS. 10A-10H.

The process 2100 displays (2138) a data visualization based onaggregating the set of tuples at the selected aggregation level to forma set of aggregated tuples. When one or more filter conditions isapplied, each tuple that satisfies all of the filter conditions isincluded (2140) in an aggregated tuple, and each tuple that fails one ormore of the filter conditions is not included (2140) in an aggregatedtuple. The process 2100 displays (2142) each aggregated tuple as avisible mark, such as a bar in a bar graph, text in a text table, dotsin a scatter plot, and so on. When one or more aggregate filterconditions are applied, aggregated tuples that fail the aggregate filterconditions are not displayed (2144) in the data visualization.

Although a filter condition may use a relation, the relation itself isnot necessarily displayed in the selected data visualization. Therefore,in some instances, the process 2100 uses (2146) a relation betweentuples to filter the displayed set of aggregated tuples withoutdisplaying a representation of the relation itself.

FIGS. 22A-22B provide a flowchart for a process 2200 for sorting (2202)data in data visualizations. Some aspects of this process areillustrated above with respect to FIGS. 7D, 8F, 8M, and 14. The process2200 is performed (2104) by a computing device 102 having one or moreprocessors and memory.

The process 2200 retrieves (2206) a set of tuples from a databaseaccording to user selection, where each tuple includes a same set offields. In some implementations, all of the tuples have (2208) the samestructure, including the number of fields in each tuple, the order offields in each tuple, the data types of the fields, and the field names.In some implementations, some of the tuples include additional fields,or have the fields arranged in a different order. In someimplementations, the fields in the tuples do not have field names, andare identified based on the order within the tuples (e.g., the firstelement of each tuple corresponds to the same data field from the datasource). In some implementations, the data types of corresponding fieldsin the tuples are identical (e.g., the first field in every tuple is adouble precision floating point number). In some implementations, thedata types of corresponding fields in the tuples are not necessarilyidentical, but are instead required to be compatible (e.g., the secondelement in every tuple is either a 32-bit integer or a 64-bit integer).

The process 2200 identifies (2210) a relation 240 between tuples. Arelation is a well-defined rule that specifies whether a pair of tuplesis related. A relation can also be considered (2210) a non-empty set ofordered pairs of tuples from the set of tuples. The set of ordered pairsidentifies the tuples that are related. FIG. 4B identifies some types ofrelations and how relations can be classified. As noted previously, theterms “relation” and “relationship” may be used interchangeably.

In some instances, the relation 240 is (2212) an equivalence relation416. In this case, the relation corresponds to (2212) a field fin theset of fields. The relation consists of ordered pairs of distinct tuples(t₁, t₂) for which t₁ and t₂ have a same field value for the field f.

In some instances, the relation 240 is a delta-tolerance relation 418.In this case, the relation corresponds to (2214) a field fin the set offields and a positive number δ. The relation consists of ordered pairsof distinct tuples (t₁, t₂) for which f field values f₁ and f₂corresponding to tuples t₁ and t₂ satisfy |f₁−f₂|<δ.

In some instances, the relation 240 is a first-order relation 410. Inthis case, the relation corresponds to (2216) a first field f and asecond field g that are both in the set of fields. The relation consistsof ordered pairs of distinct tuples (t₁, t₂) for which the f field valuefor t₁ equals the g field value for t₂.

The process receives (2218) user selection of the relation 240 tospecify a dimensional position of visual marks corresponding to thetuples. Typically, the dimensional position is (2220) either thex-position or the y-position. For example, as illustrated in FIG. 8M,the user has selected the relation {ID=father_id} to specify they-position of vertical marks (in the rows field 952). In some instances,the dimensional position is the angular position in a circular layout.

The process 2200 displays (2222) a data visualization with each tuplerepresented by a visible mark. Many examples have been provided above,including FIGS. 5A, 7B-7D, 8A-8M, 9, 10A-10H, 11A-11J, 12A-12F, and 13B.

The dimensional position of each displayed visual mark is (2224) basedon a network traversal of the tuples using the relation. As explainedabove, the relation 240 creates a graph, where each tuple is a node andeach pair of nodes that are related corresponds to an edge. If therelation is symmetric, the resulting graph may be considered undirected.There are many options for traversing the created graph, but twospecific traversal techniques are commonly used. Some implementationsuse (2226) a depth first traversal of the tuples using the relation.Some implementations use (2228) a breadth first traversal of the tuplesusing the relation. In some implementations, a user may specify fieldsin tuples or functions of those tuples to use when a traversal has tomake an arbitrary decision (e.g., which child to traverse next). In thisway the user can impose some additional order on the traversal process,which may determine how the data is ultimately displayed in a datavisualization.

Although the processes 1900, 2000, 2100, and 2200 have been describedseparately, one of skill in the art recognizes that the processesrepresent inventive aspects that can be applied together.

The terminology used in the description of the invention herein is forthe purpose of describing particular implementations only and is notintended to be limiting of the invention. As used in the description andthe appended claims, the singular forms “a,” “an,” and “the” areintended to include the plural forms as well, unless the context clearlyindicates otherwise. It will also be understood that the term “and/or”as used herein refers to and encompasses any and all possiblecombinations of one or more of the associated listed items. It will befurther understood that the terms “comprises” and/or “comprising,” whenused in this specification, specify the presence of stated features,steps, operations, elements, and/or components, but do not preclude thepresence or addition of one or more other features, steps, operations,elements, components, and/or groups thereof.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific implementations. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theimplementations were chosen and described in order to best explain theprinciples of the invention and its practical applications, to therebyenable others skilled in the art to best utilize the invention andvarious implementations with various modifications as are suited to theparticular use contemplated.

What is claimed is:
 1. A method of generating data visualizations,comprising: at a computing device having one or more processors andmemory: retrieving a set of tuples from a database according to userselection, each tuple comprising a same set of fields; identifying arelation between tuples, the relation comprising a non-empty set ofordered pairs of tuples from the set of tuples; receiving user selectionof a base tuple from the set of tuples; forming a filtered subset oftuples consisting of the selected base tuple and those tuples that areconnected to the selected base tuple by a sequence of tuples that arerelated by the relation; receiving user selection of an aggregationlevel consisting of one or more fields from the same set of fields; andgenerating and displaying a data visualization by aggregating thefiltered subset of tuples at the selected aggregation level to form aset of aggregated tuples, and displaying each aggregated tuple as avisible mark.
 2. The method of claim 1, wherein a respective tuple isconnected to the selected base tuple when there is a non-negativeinteger n and a sequence of tuples t₀, t₁, . . . , t_(n) with t₀=therespective tuple, t_(n)=the selected base tuple, and (t_(i-1), t_(i)) isin the relation for i=1, 2, . . . , n.
 3. The method of claim 1, whereinn is not greater than a fixed positive integer N.
 4. The method of claim1, wherein a respective tuple is connected to the selected base tuplewhen there is a non-negative integer n and a sequence of tuples t₀, t₁,. . . , t_(n) with t₀=the selected base tuple, t_(n)=the respectivetuple, and (t_(i-1), t_(i)) is in the relation for i=1, 2, . . . , n. 5.The method of claim 1, further comprising receiving user selection of anaggregate filter condition that is applied to the aggregated tuples andbased on the relation, wherein aggregated tuples that fail the aggregatefilter condition are not displayed in the data visualization.
 6. Themethod of claim 5, wherein the aggregate filter condition limits the setof aggregated tuples to those with at least a minimum number ofconnections to other aggregated tuples.
 7. The method of claim 1,wherein the relation consists of ordered pairs of distinct tuples (t₁,t₂) for which t₁ and t₂ have a same field value for user selected fieldfin the set of fields.
 8. The method of claim 1, wherein the relationcorresponds to a field fin the set of fields and a positive number δ,and the relation consists of ordered pairs of distinct tuples (t₁, t₂)for which f field values f₁ and f₂ corresponding to tuples t₁ and t₂satisfy |f₁−f₂|<δ.
 9. The method of claim 1, wherein the relationcorresponds to a first field f and a second field g, both in the set offields, and wherein the relation consists of ordered pairs of distincttuples (t₁, t₂) for which the f field value for t₁ equals the g fieldvalue for t₂.
 10. A computer, comprising: one or more processors;memory; and one or more programs stored in the memory configured forexecution by the one or more processors, the one or more programscomprising instructions for: retrieving a set of tuples from a databaseaccording to user selection, each tuple comprising a same set of fields;identifying a relation between tuples, the relation comprising anon-empty set of ordered pairs of tuples from the set of tuples;receiving user selection of a base tuple from the set of tuples; forminga filtered subset of tuples consisting of the selected base tuple andthose tuples that are connected to the selected base tuple by a sequenceof tuples that are related by the relation; receiving user selection ofan aggregation level consisting of one or more fields from the same setof fields; and generating and displaying a data visualization byaggregating the filtered subset of tuples at the selected aggregationlevel to form a set of aggregated tuples, and displaying each aggregatedtuple as a visible mark.
 11. The computer of claim 10, wherein arespective tuple is connected to the selected base tuple when there is anon-negative integer n and a sequence of tuples t₀, t₁, . . . , t_(n)with t₀=the selected base tuple, t_(n)=the respective tuple, and(t_(i-1), t_(i)) is in the relation for i=1, 2, . . . , n.
 12. Thecomputer of claim 10, wherein the one or more programs further compriseinstructions for receiving selection of an aggregate filter conditionthat is applied to the aggregated tuples and based on the relation,wherein aggregated tuples that fail the aggregate filter condition arenot displayed in the data visualization.
 13. The computer of claim 12,wherein the aggregate filter condition limits the set of aggregatedtuples to those with at least a minimum number of connections to otheraggregated tuples.
 14. The computer of claim 10, wherein the relationconsists of ordered pairs of distinct tuples (t₁, t₂) for which t₁ andt₂ have a same field value for user selected field fin the set offields.
 15. The computer of claim 10, wherein the relation correspondsto a field fin the set of fields and a positive number δ, and therelation consists of ordered pairs of distinct tuples (t₁, t₂) for whichf field values f₁ and f₂ corresponding to tuples t₁ and t₂ satisfy|f₁−f₂|<δ.
 16. The computer of claim 10, wherein the relationcorresponds to a first field f and a second field g, both in the set offields, and wherein the relation consists of ordered pairs of distincttuples (t₁, t₂) for which the f field value for t₁ equals the g fieldvalue for t₂.
 17. A non-transitory computer readable storage mediumstoring one or more programs configured for execution by a computerhaving one or more processors and memory, the one or more programscomprising instructions for: retrieving a set of tuples from a databaseaccording to user selection, each tuple comprising a same set of fields;identifying a relation between tuples, the relation comprising anon-empty set of ordered pairs of tuples from the set of tuples;receiving user selection of a base tuple from the set of tuples; forminga filtered subset of tuples consisting of the selected base tuple andthose tuples that are connected to the selected base tuple by a sequenceof tuples that are related by the relation; receiving user selection ofan aggregation level consisting of one or more fields from the same setof fields; and generating and displaying a data visualization byaggregating the filtered subset of tuples at the selected aggregationlevel to form a set of aggregated tuples, and displaying each aggregatedtuple as a visible mark.
 18. The computer readable storage medium ofclaim 17, wherein a respective tuple is connected to the selected basetuple when there is a non-negative integer n and a sequence of tuplest₀, t₁, . . . , t_(n) with t₀=the selected base tuple, t_(n)=therespective tuple, and (t_(i-1), t_(i)) is in the relation for i=1, 2, .. . , n.
 19. The computer readable storage medium of claim 17, whereinthe one or more programs further comprise instructions for receivingselection of an aggregate filter condition that is applied to theaggregated tuples and based on the relation, wherein aggregated tuplesthat fail the aggregate filter condition are not displayed in the datavisualization.
 20. The computer readable storage medium of claim 19,wherein the aggregate filter condition limits the set of aggregatedtuples to those with at least a minimum number of connections to otheraggregated tuples.